{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d8bfe72e",
   "metadata": {},
   "source": [
    "# Hyperparameter tuning and lags selection\n",
    "\n",
    "Hyperparameter tuning is a key step in building accurate and robust machine learning models. Hyperparameters are configuration values that cannot be learned directly from data and must be defined by the user before training. These values can significantly affect model performance, and carefully tuning them helps improve both accuracy and generalization.\n",
    "\n",
    "In forecasting models, the selection of **lags** (past time steps used as predictors) is considered an additional hyperparameter, as it directly influences the model's input structure and learning capacity.\n",
    "\n",
    "Hyperparameter tuning consists of systematically evaluating combinations of hyperparameters (including lags) to find the configuration that yields the best predictive performance. The **skforecast** library supports several tuning strategies: **grid search**, **random search**, and **Bayesian search**. These strategies can be used with either [backtesting](../user_guides/backtesting.html) or [one-step-ahead validation](../user_guides/hyperparameter-tuning-and-lags-selection.html#one-step-ahead-validation) to determine the optimal parameter set for a given forecasting task."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6bd6f81b",
   "metadata": {},
   "source": [
    "<div class=\"admonition note\" name=\"html-admonition\" style=\"background: rgba(0,184,212,.1); padding-top: 0px; padding-bottom: 6px; border-radius: 8px; border-left: 8px solid #00b8d4; border-color: #00b8d4; padding-left: 10px; padding-right: 10px;\">\n",
    "\n",
    "<p class=\"title\">\n",
    "    <i style=\"font-size: 18px; color:#00b8d4;\"></i>\n",
    "    <b style=\"color: #00b8d4;\">&#9998 Note</b>\n",
    "</p>\n",
    "\n",
    "All <b>backtesting</b> and <b>hyperparameter search</b> functions in the <code>model_selection</code> module include the <code>n_jobs</code> argument, enabling <b>multi-process parallelization</b> to improve computational performance.\n",
    "\n",
    "Its effectiveness depends on factors like the estimator type, the number of model fits to perform, and the volume of data. When <code>n_jobs</code> is set to <code>'auto'</code>, the level of parallelization is automatically determined using heuristic rules designed to select the most efficient configuration for each scenario.\n",
    "\n",
    "For more information, see the guide <a href=\"../faq/parallelization-skforecast.html\">Parallelization in skforecast</a>.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68a589bf",
   "metadata": {},
   "source": [
    "## Validation strategies\n",
    "\n",
    "Hyperparameter and lag tuning involves systematically testing different values or combinations of hyperparameters (and/or lags) to find the optimal configuration that gives the best performance. The **skforecast** library provides two different methods to evaluate each candidate configuration:\n",
    "\n",
    "+ **Backtesting**: Simulates a real deployment scenario by generating multi-step forecasts in repeated iterations, using the defined forecast horizon and retraining frequency. This approach provides a realistic estimate of performance over time. Use the <code>[TimeSeriesFold](../api/model_selection.html#skforecast.model_selection._split.TimeSeriesFold)</code> class for this validation strategy. [More information](../user_guides/backtesting.html).\n",
    "\n",
    "+ **One-Step-Ahead**: Evaluates model performance using only one-step-ahead forecasts ($t+1$). This method is faster, as it requires fewer iterations, but it only tests the model's performance in the immediate next time step. Use the <code>[OneStepAheadFold](../api/model_selection.html#skforecast.model_selection._split.OneStepAheadFold)</code> class for the one-step-ahead strategy. [More information](../user_guides/hyperparameter-tuning-and-lags-selection.html#one-step-ahead-validation).\n",
    "\n",
    "Although the two methods may produce different results, they tend to converge on similar hyperparameter selections over time. The one-step-ahead method is faster than backtesting because it requires fewer iterations; however, it only tests the model's performance in the next immediate time step. For a more accurate multi-step performance estimate, it is recommended to backtest the final model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76193903",
   "metadata": {},
   "source": [
    "<div class=\"admonition note\" name=\"html-admonition\" style=\"background: rgba(0,184,212,.1); padding-top: 0px; padding-bottom: 6px; border-radius: 8px; border-left: 8px solid #00b8d4; border-color: #00b8d4; padding-left: 10px; padding-right: 10px;\">\n",
    "\n",
    "<p class=\"title\">\n",
    "    <i style=\"font-size: 18px; color:#00b8d4;\"></i>\n",
    "    <b style=\"color: #00b8d4;\">&#9998 Note</b>\n",
    "</p>\n",
    "\n",
    "For a more detailed comparison of the results (**execution time** and **metric**) obtained with each strategy, visit <a href=\"../faq/parameters-search-backtesting-vs-one-step-ahead.html\">Hyperparameters and lags search: backtesting vs one-step-ahead</a>.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "fc006084",
   "metadata": {},
   "source": [
    "## Libraries and data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "902da042",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Libraries\n",
    "# ==============================================================================\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from lightgbm import LGBMRegressor\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from skforecast.datasets import fetch_dataset\n",
    "from skforecast.recursive import ForecasterRecursive\n",
    "from skforecast.plot import set_dark_theme\n",
    "from skforecast.model_selection import (\n",
    "    TimeSeriesFold,\n",
    "    OneStepAheadFold,\n",
    "    grid_search_forecaster,\n",
    "    random_search_forecaster,\n",
    "    bayesian_search_forecaster\n",
    ")\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f7ad54d0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">╭────────────────────────────────────── <span style=\"font-weight: bold\">h2o</span> ───────────────────────────────────────╮\n",
       "│ <span style=\"font-weight: bold\">Description:</span>                                                                     │\n",
       "│ Monthly expenditure ($AUD) on corticosteroid drugs that the Australian health    │\n",
       "│ system had between 1991 and 2008.                                                │\n",
       "│                                                                                  │\n",
       "│ <span style=\"font-weight: bold\">Source:</span>                                                                          │\n",
       "│ Hyndman R (2023). fpp3: Data for Forecasting: Principles and Practice(3rd        │\n",
       "│ Edition). http://pkg.robjhyndman.com/fpp3package/,https://github.com/robjhyndman │\n",
       "│ /fpp3package, http://OTexts.com/fpp3.                                            │\n",
       "│                                                                                  │\n",
       "│ <span style=\"font-weight: bold\">URL:</span>                                                                             │\n",
       "│ https://raw.githubusercontent.com/skforecast/skforecast-                         │\n",
       "│ datasets/main/data/h2o.csv                                                       │\n",
       "│                                                                                  │\n",
       "│ <span style=\"font-weight: bold\">Shape:</span> 204 rows x 2 columns                                                      │\n",
       "╰──────────────────────────────────────────────────────────────────────────────────╯\n",
       "</pre>\n"
      ],
      "text/plain": [
       "╭────────────────────────────────────── \u001b[1mh2o\u001b[0m ───────────────────────────────────────╮\n",
       "│ \u001b[1mDescription:\u001b[0m                                                                     │\n",
       "│ Monthly expenditure ($AUD) on corticosteroid drugs that the Australian health    │\n",
       "│ system had between 1991 and 2008.                                                │\n",
       "│                                                                                  │\n",
       "│ \u001b[1mSource:\u001b[0m                                                                          │\n",
       "│ Hyndman R (2023). fpp3: Data for Forecasting: Principles and Practice(3rd        │\n",
       "│ Edition). http://pkg.robjhyndman.com/fpp3package/,https://github.com/robjhyndman │\n",
       "│ /fpp3package, http://OTexts.com/fpp3.                                            │\n",
       "│                                                                                  │\n",
       "│ \u001b[1mURL:\u001b[0m                                                                             │\n",
       "│ https://raw.githubusercontent.com/skforecast/skforecast-                         │\n",
       "│ datasets/main/data/h2o.csv                                                       │\n",
       "│                                                                                  │\n",
       "│ \u001b[1mShape:\u001b[0m 204 rows x 2 columns                                                      │\n",
       "╰──────────────────────────────────────────────────────────────────────────────────╯\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>datetime</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1991-07-01</th>\n",
       "      <td>0.429795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1991-08-01</th>\n",
       "      <td>0.400906</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1991-09-01</th>\n",
       "      <td>0.432159</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   y\n",
       "datetime            \n",
       "1991-07-01  0.429795\n",
       "1991-08-01  0.400906\n",
       "1991-09-01  0.432159"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Download data\n",
    "# ==============================================================================\n",
    "data = fetch_dataset(\n",
    "    name=\"h2o\", raw=True, kwargs_read_csv={\"names\": [\"y\", \"datetime\"], \"header\": 0}\n",
    ")\n",
    "\n",
    "# Data preprocessing\n",
    "# ==============================================================================\n",
    "data['datetime'] = pd.to_datetime(data['datetime'], format='%Y-%m-%d')\n",
    "data = data.set_index('datetime')\n",
    "data = data.asfreq('MS')\n",
    "data = data[['y']]\n",
    "data = data.sort_index()\n",
    "data.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8f18cddd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train dates      : 1991-07-01 00:00:00 --- 2001-01-01 00:00:00  (n=115)\n",
      "Validation dates : 2001-02-01 00:00:00 --- 2006-01-01 00:00:00  (n=60)\n",
      "Test dates       : 2006-02-01 00:00:00 --- 2008-06-01 00:00:00 (n=29)\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAo8AAAExCAYAAAAduEjjAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAnupJREFUeJztnQWcHPX5xp/Z3XPXnMSFKEmAIKE4BVrcWry4/4sUKQ7FaxRatNDiWqxQ3DUBkpAQd78k524r8/+8M/Pbnd3b3VsZ2bt7v3zC3u3t7fx2Zm7n2ec1CeW7yWAYhmEYhmGYGHDE8iCGYRiGYRiGIVg8MgzDMAzDMDHD4pFhGIZhGIaJGRaPDMMwDMMwTMyweGQYhmEYhmFihsUjwzAMwzAMEzMsHhmGYRiGYZiYYfHIMAzDMAzDxAyLR4ZhGIZhGCZmBq14dEoSCrOylNtUgNcTHV5PdHg90eH1RIfXEx1eT3R4PdFxDsH1DF7x6HCgKDtbuU0FeD3R4fVEh9cTHV5PdHg90eH1RIfXEx3nEFxParxShmEYhmEYZkDA4pFhGIZhGIaJGRaPDMMwDMMwTMyweGQYhmEYhmFihsUjwzAMwzAMEzMuDED23GUyfjZrmvK1HOExVKCekeZCj9sT8TFWwuuJvA5i7vyl2LZxh40rYRiGYRhmUIrHXx15ADweLx7492vwen1RRUma0wm315syYo3XEx6n04FjDvkZpo4bhadf/9Dm1TAMwzAMM6jC1iOqyvHG+19FFY7MwIKOJR3T8vJiu5fCMAzDANgrV8bYDLutBSZVGXDiUZb5ZB6s+GT+QMAwDGM3U7NkfDHNhzcm8XsyM0jEI8MwDMMw5nF8iQynBEzNBrIdbNgwfWHxyDAMwzCMnyOKAoJxfKatS2FSFBaPA5Dv3n0C5512tN3LYBiGYQYZ1ekyZuUGvmfxyAyKauuByn+euBsrVm3AzX/6Z9LP9cvTfoeurh5D1sUwDMMw4VxHYkKmDLTYthwmRWHnMYWgljWx0NjUiq5uFo8MwzCMsRylicd6t/r9+Cx718OkJoNDPKal9/kn6/7B6H9x8rfbr8Des3bGOaceha0L30bNwrfx66MPVm4P/NlueP+Fv2HDD69jj12mYNTwCjz5txux6JNnsPrbV/Du8/dh3z1nRA1b0/Ocetyh+Nd9N2DtnFfx9VuP4dD99zBk1zIMwzBDgxyHjAML1K8f2yEFnMcUItMho9iVWmsaigyOsPXVT4S9W/vgZDz3/Cauh9/yp39i3KgqrFq7GX966FmlKffEcSOVn91w2Zm442//xsYt29HS2o6qilJ88vV83Pvgs+h1u3HikQfhqQduxn7HXoyt2+sibuN3F56MO+9/Cnf87Umcc8qRePDuq7DHL89Fc2t70i+XYRiGMZeR6TKavUCrV8zdsp6fF5I4A9Z2A281SrhxuJxyOY8H5gNvT3bj284m/LzZvn2lZ2QpkDXEBO3gcB5TnLb2TvS6Peju7kFdQ7Pyz+tT+2f95ZHn8eXchYp4JKG3bNUGPPfa+1i5dhPWb9qGPz/8vPKz/pzEl9/6BG++/yU2bN6Ge/7+DHJzsjFz2k4WvUKGYRgmUcrTZKzYxYcPpvhSImT9v0YJq7vV+yrSgbwUatezc7a6lgZPasiXYQXAhzdJuO/sDqQKP98ZGF/pNXUbg8N5/Mt5Ye8W4/dSmUXLVgd9n52ViasvOhUH7zsL5WVFcDmdyMxIR3VlWdTnWb56g/9ryodsbetAabEWf2AYhmFSlp0ygXQHsGsOkCbZI9QckP3FMm83SYoDWusmYQuMy5QROe5lLTvnqLcrelxmxhdjZnK1hIw0CZUhhUZ2MXUE8PhFDmyq68L+t6WQeNxz16m45MzjsfPkcagoL8E5V96F9z+bG/HxvzxoNs789S8xdaexSE9PUxy1vz76Ar6Y8yMMw93b5y4ysyWfE1IKzG6ORmdI1fQtvzsH++01E7ff92/FRezu6cXjf7kOaWnRDxXN+9YjQ4bDkRqfzBiGYZjIFGtv79SYe1QGsMkGTbRPPlCWBjR5gK/b1PvWdOnEY4oMm5mmOY+relNDPA4vUUPnrhRxZw/ZWb3uF+aYe8DiVhfkjC1dtR433PNoTI/fa7epSlj29N/+Ab849Qp8O+8nPP33mzFt4lgMJdxuT0xibveZk/HKW58ognzFmo2orW/C8KpyS9bIMAzDWE9JWkB4jM2wZw2nlKpreKNRgkdWBdHqbvU2VfIe0yUZkzL1zqP9DC9Wb11OpAQHT1OPWT9+U9LE/fSffTNf+Rcrt/45uJjl3n88i8MO2AuH7L8Hlqxch6HC5podmDltgiIEOzq74ZDCC8n1m2pw+MGz8dGX3ytzvK+95PSIj2UYhmEGj/NIjMmU8Xm79aLsxBJVPL5YFyhCWaPlPZLziE7YzqQsIM2huqPbUyTnsdrvPNq9EqA4F5g5WhOPJotZy6W7JEnIzc5Cc4vmi4fBKUlwRnDplHB0jNvx38r228mPPfMmHrjjCnz+2sPIysrAlbfcH/b1/OGv/8J9t12Gt576MxqbW/HQU68iNze7z+PC7Qcpjn2VavtHrIfyVFMBsQ5eT3h4PdHh9USH1xNMmeI8qu/DE7Iky9dzdJEPRS4ftvYCczqcSKf4OZkZSkaYFxM0t8/u47VLrpqetbSL9EFgP9nJiJKA05cWY69ms/j5NMDh0MSsE8hIUNH2xlArYrl4vPjM45CdnYm3Pvw64mPyMjNRlJ0d9mcZaa64ThhXiuT9bd6yHceffV3QfW+887lyq389O3Y04LSLbw163AuvfhD0uH2PujDo+9G7HdvneWYccHqf+1J5/wjK8/KQSvB6osPriQ6vJzq8HpXh2a0AVJtvSm6afx1WreesChoh48F77dmoLAzMJmxLo5zCJn+vR7uP115FZMl2Yr03IyXWQ4wsISNM3T+VBXnw+uxrH3T4LmQPe/zfVxfloccd/3rWNzSklng87pf743cXnoKzr7gTDU2R5x21dXejs7dvEQzR4/bEVEFNThYJI4/Pp4R/7YbX0/96iNq2tpSokCfRTW9MvB5eD6+H12M2maWBwo8KR6+yDqvWU+CUccB4dfuPb+3F1q5m/89aqAhkFIXVZeQ5fFjb0mHr8Rpdrq5zXosqkOw+f1wOoDQ/EONr6mxDW7c9lUVkOO4+Pjje2NLVhoZ2c9ZjmXg85rB98ZdbfosLrr0XX323KOpjvbIMb4QTImDu94MmiEgY2S+NeD2xrofeCGKxzK2C1xMdXk90eD3R4fWoFDrloJxHt9dj2XqOLPEpjcGXdALzFaERWEujF9jWC1SmA6PTvFhh8/GaqlVaL2yXgTT7z59hhYDTkaa7x2fbemaNlVCY40Jzh4z8LDV8LTnMW48lMctjf7Ef7vvD5bjk+j/jk6/mWbFJhmEYhhlwBTN5TqDUwpjgqVqV9QtKoUzfEKcomhmdbq/IL3HJqNKmAy/rSo3JMsO1fEeBnRXXB05V1/Llcvrwod6X4UqxVj1TJ45R/hEjqocpX1dXqE2sr//tb/DAHVcGharpe+pbuGDxSpSVFCr/8nLD5zQyDMMwzFAVj8SYDGviQZVpMvbPV79+qT68IBPtekal2Ssed9YkA41O7LAxrzBcm55UEI8HTVPl3KdLfOjR0h7TTRSPcT/1jKnj8doT9/i//8PV5/nH41EFcXlZcdA0lNNOOExpcH3PDRcr/wTi8QzDMAwzlCnRIp8be9Qm4RS6rrFAqx1aKCu5ct+3AZt6wwsyahROjE4PFGLYOZZwcQq0DBJUFwfvM7Pb40Ri+kgJ00epa/lsqYzeVBSPc+YtQdXMoyL+PFQQnnjeDYmtjGEYhmEGOZkOGVlaDHBeuyYeM4BvLBBJB2sTbD9qiezkqc6jrOQ8WpTpFnUs4eIOa11H6r7j9aVu2HpiFfD8ZeqGP/rJh/o2WCIeU6tPC8MwDMMMIUq0C7zbB/ykCaOxFoStJcg4uEDdzsfNkQXZBm2CbkWaLyWcx586rROPuZnAd3e78PiFzpQMW48tB1663IXiXAkL1vvwf/9W7eqUdB4ZhmEYhjFWPDZ6dMUpFohHyiEclg60e4G5USbadPkCU2jswgEZU7PUr6kq3ComVUmoLJRQPgPIzgA6NSEdyXm0MmxN63nxchfKCyQs2yLjjH940a6dP+w8Mn6+e/cJnHfa0f7vaxa+jV8cuFfEx9MYRHqMKGxKFKOeh2EYholcLEPicX2P5jxqTbnN5Oea6/hlK+DWZlmHo1cTj2k21qicN0xGthPo9KoFM1ZRqIXKnQ4JO48I3gGSBFQVqV97fbLlzuPPdpIU8bqtWcbJD3jQrBPVvVrbUHYemT7MOPgMtLQaOwD1b7dfgYK8HJxz5V3++2q21yvbolGJDMMwjHnicZ0mjKrTzXf6fl7Yf8iacGvLSLOlI7CM31fJuGuUuu2Ht0vwxTSg2BiKNPFI0Mzo79YE9kFZHk28kxThuKNFFZJWOo9qQ3CqrpbREDLtmZ1HJiJ1Dc3odZtf/ebz+ZRteSNlDDMMwzAxkSbJeHWiF9dVB95PaXoL0eAB6j1Aq/a2PtxlXrl1hiRjH22y38dRimWIXtk+5/HekQHheO8WCddtsnYRRTmB7c3QqplDQ9bbm4FubSCeleJxD008/rCm77WZxeMggdoVzf/wKf8IPsGTf7sR9912GUYNr1C+XvTJM1j97St49/n7sO+eM6I+Z2jYeua0Cfjwpfux7rvX8N7z92HaxLFBj3c4HPjrrb/F3HeewNq5r+LLNx/B2acc6f/5VRedgpOOPlh5Tnpu+jd71rSwYeu9dpuGd577K9Z//zp+/Ohp3HDZmXDqBsK/+sTduOPaC3DTFWdh6RcvYOHHzyjPzzAMQ27S7cM9ODLPwvhjirBvHnBsMfD76sCstEDOo9qke52WVzfSxKbce+dBCQPX9FLDbcTkPFJLH8o9tAqap321sp+AqzZIuGkzXWOsFY8ibC2cx3DFMlsaAk25rQpbZ6YFxOz3Ojc0VDymVJPwVCTbIVv6L17+9+HXKCrMx+xZO/vvK8zPxQE/2w2vv/s5crIz8cnX8/HrC27CoSdfjs++mY+nHrjZ33i939eflYln/n4LVq3bjF+ceiX++uiLuOV35wQ9hkYVbattwAXX3IsDjr8Uf/vnS7jm0tNx1KH7KD9/5Ok38NYHX+HTr+crYWr6N2/hij7bqigvxnMP3opFS1fjkF9fhuvvfgSnHHsIrjj/pKDH/eqog9DZ1Y0jz7gKd97/JK684GTst9fMuPcdwzCDi5nZwNVVPlxbamzazUBgZo7snyIjwtXFaYGwNbFe09QjTGzKLULWnyiuYz/Oo87YSrdQu43PVG9/7AAe2GaPVNGHrUeVSUHfV2vO45ZGwOO11nmkno7pLgk7WmRsrO/7c662jpHWPSOFVE0aCD4nvjOkpa1DEYTH/GI/fDH3R+W+Iw75mZJH+M0Pi5X50stWbfA//s8PP49fHjQbh+6/B558+Z1+n/+4w/eHQ3Lgqtv+jp5eN1at3YTKYSX4402X+h/j8Xjxl0de8H+/pWYHdp8xGUcdsg/e+vBrReh19fQiPT1NCVNH4sxfH6HkQd5wz6PK92s2bMGwsmLcePmZuO+xl5TXQixfvUH5nli/aRvOPvlI7LPHDHw5d2Fc+45hmMEF5fOJ/oZDjRk68TE6QxWM+pxHYp1SNCNjpIniMdCip//HirA1kW6hhqtMVzdM7qhdFOrC1gS5fZ8vU9clnMetjTLGD5MsdR53HydC1uH/hvziUT9222AGhXgcCLzx7uf48y2/xXV3PYQetwfH//IA/Pf9LxWxRc7h1RedioP3nYXysiK4nE5kZqQHTeqJxoQxI7Bs9XpFOArm/9TXNTzrpMNx8jGHKI5mZma6Mvln6cr1cb2OCWOG93nuHxYuQ25ONqqGlWLr9jq/eNRTW9eI0mKtIy3DMEOWYZoosLN6127nkaBm4As6dDmP2tv3OpOdxyKXjF01Eas6j7GFra0+ZmKOdU2EyTdWIJzGrl4ZWemSEroOiEfJtrD1HkI8ru1HPLLzGJ387/p+HKJdm+Z0wu312lIjFspHX3yvLOrgfXfHwqWrseeuU3DbX55QfkYhZgrp0vzvDZu3obunF4//5TpF3BnFMYfti5uvPEfZxvxFK9DR2YVLzzoBM6ZNgBm4PcHFPHQMJMegyJJgGCYJKjQ3JD0l3pmtg5zWSVqvwkAvR8mf89ig5DxS2FrvPBqvRiZlqvmLNApxu7t/YSZDgkcGXJK1YetK7TzZlgLO43erZRwwVQoqmqm2KWxNpROzxkXOdyRScrZ1KtIZZki6Ih4lCW4f9dG3H3IFP/h0Lo47/ACMHlmJtRu2YvGKtcrPdp85Ga+89Qne/2yu8j05kVSoEiur12/GiUcciIz0NL/7uOvOk4IeQ9uYt2gFnn7lXf/+GTm8Iugxbrcbzn4E3ur1W3DEwXuHPPcUtLV3omZHmOQLhmEYHeWaKCAxIopGhgLUlNupu1SR80gUaVfhJu2CX6M5kMVOSrsyXo2Ua47e9jhEGbmPdLzUY2YNVSkQthbOI7mNB0wFZuiKZobbVDCzU6Uqaju6ZSzdEt15TNOfcAbDVpCFvPneF0pomkLHVCgjWL+pBocfPFupaJ6y02g8fM/VSg5jrLzx7hdK+JvC4hPGjsBB++yGi848LugxlHc4Y8p47D97F4wdWYVrLjkN06eOD3rM5ppaTJ4wGuNGVaO4MB+uMH8JT7/yDqoqSnHXdRdi/OjhOOyAPZWQ+z+fe9Of78gwDBOJCk0UkPtlwyhg25ihjdcTiCkyAedRve3xT3QxZx3D0tTt7ghkOfWLKJpJtyFsvc3GsLWotv5quQ8er4xhBTRxBijIBvKy1HVtbYRfPFrhPO4xTtUGC9bLEWduc6ueQca3PyxGc0sbxo8Zjjfe+8J//21//ReaW9vx1lN/xtMP3ILP5yzwu5KxQMUuZ15+ByZPGIUPX3oA1/3fGbjr/qeCHvPsq+/hvU/n4NE/XYv/PfdXpfr7uf+8F/SY51//AGs3bsV7L9yHJZ8/r7iVoWyvbcTp//cHzJy2Ez565e+498ZL8OKbH+H+x19OaJ8wDDM0w9ZDLe9xZk6gepgYpVQTy0HjCYPFo2yq87sjhpB1aNFMuoVFTkI8brXJeSQhmJup7qNtzcDKGvX+XcZIuPF4VSXSdJdut7Vha1Es832EfEeCcx4HGeTM7XboWX0CNVtqapU2PXqeelkNLwv2PPy8oO+rZh4V9P2CxStxyEmXR3wMNRS/8tYHlH/6nNC7Hnja/5jGplaccvEtfdYduq2585fgiNOvivg6Tzzvhj736afWMAwzdBmWZk/1bqoUy7zVKGGXHFmptqaWPWmOEOdRu0BkKPfLponH2jicR8p5tFLsOyH7zxMRxrfLdaQJMq1dwMKNMqaOkPDH05wozlUny9zxqqoaPSaHraeOAMrzJTS2A3tOiF4sY1WfRxaPDMMwjOHkOGTM2dmHOW0SLlwXUIkVmqNkdRjUTqi5NuU8CvF46whZEY4TtF6GXT76JwU5j0Ks9aZC2Fq29ngNS1fTGki01rntzXds6STjB1i0wYfT9nEowtHtlfHbf3vx9nx1x5iZ80g5ju9e54JLl79IwnXBumjiUS3G4rA1wzAMM6DYMxeYkg2cWhaYpkKCkkRTKoStyf2r1vIvzYZEYo4T6PACizsDRSC7am6kCFnrnUezxJpw9HbEkUso2vVYVTBTpau0pmpvOxCjCZu1NAMRJu5xyzj/sYBwJMwMW5+6j0MRjg1tstJTsrNHxsvfyujQJhGFg/s8MgzDMAOS8VnqxTXLoeY5bncHh6ztFI8j0mV8O82nrGnKQgfMa8cdHLIm4eiDpLTJoZy+XXMR1OMx1HlUQ9f2h617FVdUtsx5rPQXy8A2RNi6qUM9dqu3AWc+5MG2JqpyDn6sWWFrEqMn7KmeBFc87cWnS2L7sMMFMwzDMMyAZCctJEuMyegbsrZTPNKkF8o1HJEBXDhMtqxYZmGH+oI3KFNkVPdT36aHIHEpcgwzzHQeUyjn8bBCGVO0DxuEcITtKpbRh62F80h8vLivcCTcPnPE4yHTJSVMToU5X2jNyWNBzAth8cgwDMMMKMZnBi52o7WvQ51HK6t3w7XJIa6tlpFt8jqE87hQEyIbtSky07ODi2UEZrXryZBkFGiCIqGcR4c558k7k314c5Kv73SZOCrCzWoQ3qQTj5Fwi76KlKhpICfvre7wV+f6IrblCQc7jwzDMMyAZEJWGOdRK9awu2BGrEeEcS8sj+PKHDeyf6Z1wHkMDkuL6TKC4Ipr4xDincRpizc1CmamaQJ6bCZQpo1qTIWwdcB57P+DhceEsPWwAihTbYhXvo3v/LRiwgyLR4ZhGMbwVitjdQJtdGZqha3HaE7onDb1+ysrvciRfKaN2SOBSqHfJV3qfRu1sLVAH7bWh0HJKTS6ijngOiZSMGO8QztB51ALkZ0K02Xich696q2BE4Vx4l4OOB0Svlvtw7ra+H6XnUeGYRhmwEFiUfQvJMZoYeJUKZihHovEPVscWNUFlKYBZxRpys5gxBjCzT1Aty/YeRT0CVub7DzGE7I223kcp8uNnaGF9wNzrSXbncdYxKPHhGprEbJ+eU78H2pYPDIMwzADtljGKweLtdQIW8sYo61vTTdw1xZ1Eb/KN0c8igky9TqBSNXWevStevTi0ej9U67t/3gqrc0umNHnxooc0GqR85gCYWtRbR0Nj3aiGxW2nlAJjB0moatXxv90LYFihcUjwzAMM+AQguCHdvV7qmpWpoaEhq1tuAKVutTJLj5ZFXELtDzEPKc5RTPFmmDTC8ReWQqqJG4IKQzp0RxK85zH+FSgmbOt9c7j9BxZCdWXaOvcOsDC1i6DxGN1kbrtdTsQtZ9jJFg8DiL+88TduOWqcw17vr/dfgX+/bcbDXs+hmEYo4tlvmqV0O1Tm0uTgBRzrUVOnx3Oo3AdydXqkSV/ZbNZIXThPIYKRFFxHc557DWpVY+/x2OcoozErhliP8shK+eFYHJW4PjQ1J1msxtwGlUw4zM2bD2sUL3d3pLYBxoWjwzDMMyAQxRBrOgKhGipgCZ0XrEdOY8i/1LkHQby+WRzxWOIQBS9HsOGrUXBjEnOY7xha1EwY/TxEkVVzR61UTp9yDi4QF8sY2ernvhzHl0GHa+KAvV172hO7Pd5tvUggVzCvWftrPw759SjlPv2OPxc5GRl4uYrz8Geu05BZ1c3vpizELf95Qk0Nrcqjzni53vjdxeegtEjKtHV3YOlK9bhrCvuxCVnHY+Tjj5YeUzNwreV2xPOux5z5i2x8VUyDMOoiJnNa7olbOiRMTGLpqnI/j6BW3sljMqQkWaSYIulTc+67uBZ0qoLavx6il3hBaI+7zFSwYzRgjaRudZ68Wi0UyxC1qu7gXYvcGAB8ItC2fY2PZlpQJb2YvVNwq0KWw/TnMcdKew8DgrxmBWSR0NImoXs8hr/dtAV50l9y5/+iXGjqrBq7Wb86aFnlfV4PF68+9xf8cIbHyqCMTMjHTdecRYe/dO1+PUFN6G8tAgP33MN7nzgKbz36RzkZmdhz12nQpIkPPL0G5gwZgRyc7Jx5a33K9tobtGSixiGYWyEctZGagKNKpnXKyJNxl556jtxvVud8WxX2Fq0DQp1HoWzZrRmKdEEW1/nMUrY2iTnMTCaUEoJ51HkxtKHDAqlH1gg44AC9Wc1KVBp7fbKaNelF/TbJNxpzPaHac5jbUtiv8/iMUbW/D3N0sh89UXxfWxra+9Er9uD7u4e1DU0K+Lx8vN+jSUr1uHefzzrf9zvbn0A8z98CmNHViEnOwtpaS68+8m32LqtTvn5ijUb/Y/t6ulFenqa8nwMw9iMwwmceAVQsxb4+k0MZchNokEbLR6gzhMQSXtpc5xpnrRZYiQWxmph6/U9wS6fyDGMwWhKMOcx+H6116OMNi/tj/BNwo0W18F9HhMpmKGFGbeo8Vpu7NpucoLhn4VuR6X1xYc6sLJGVuZH+yutY/RkPAb3eSwvSC7n0Yom4YNCPA5Epkwcg7133xmrv32lz89GjajEF3N+xFdzF+LT/zyIz+csUL5/56Nv0NJm9FsbwzBJUzkWGDcDGDkJ+Pq/GMqIYplVihhQw9b6BuE7es0ddxer86g6ogFhZNZ6AmHrYNH1YwcU4TgvjEBRq60DYX4joBQBsZZE+zwaXTDjdx67gMWd6msWWCked6oEbjreqRTH7Hy1x19p3dwZ2++7DQ5bJ53zqB1fh0NS8jBFQY+RDArxOP4yd4SwtRNur9eELJbkycnKwkdf/IC7Hniqz8921DXC5/PhpItuxu4zJ2P/2bvgnJOPxHX/dwaOOP1qbK7ZYcuaGYaJQEGpepuWAeQXAx0pFBEoKIWvaizkuvWWFstQKJLYoIWtBdQmxqXMAKacR1gKtQsaqYlY4Tz6ICn9KJ2S8dXN0QpmaCTh6PkOdIS5sPubhBu4nrK0QM/G0DB5f5jlFIucx7XdEpZ3qVX4QqCKoiorEGFiEo0Tq4AizSVvao9NPXgMrLaWpIDzmGzOI5FO040SaPczJMRjuBxEOhU8TvUTQSqIR7fbA4cj8LFt8Yq1OOLgvRUh6I0y8fyHhcuVf/c99hK+f+9f+OVBe+Gfz/0XbrcbTt3zMQxjIwUlga9LKlNLPO76c3j2Ohx1q+cDbzxoWbHM6q5gkSYg16tcE3BWi8fhGao4oSIZvbNF32c7zViPrEyvCSceiRZv+A36cx4lcyqt5ThDz2YUzFBu7Ij0QMEMtQMiATk9x/rpMkIsEruPc8BLTUDjcB49BlZbl+SSgynB55NRp9bOJiUeqeK60wTxyOrDIkgkzpw2AcOrylFcmI+nXn4HhQW5ePjeazBj6gSMGl6hOIx/+8PlisjcZdpO+O25v8L0KeNRXVGGww+ejZKiAqxev0V7vlpMnjAa40ZVK8/nMnIiO8Mw8ZGvOY9CPKYKzjRg+r7Kl7lrf7RkkxOyZL8gEKKJwrMCJefRpj6PotKa8jD1AsqsMHq2A8jUnpMKhWIlMJ5Qtn00ISFyMo0U12N1ubFi+s5PSuja+rB1kRamJnYfL/m/jzXn0W1g2FpUWte3AVF8paiQ9hWC1qy8RxaPFvHY028ooejPX3sYSz5/XimGOeasaxX38MVHbldyG2+/5nwlp5Ee19bRib12nYrnHrwVX/33Ufz+0tNx+1//hc++ma883/Ovf4C1G7fivRfuU56PwtsMw6SA81icQuJx0iwgOw9obUDW5mWWbDLgPIoLsoT1uorV7b3WFszkOmS4tJY3o0N6PPYNE8umhKzJ2QwXno5EcPsge0cT6p1QI4/XON2ISFGEs0iX0m+teAx8vfs4yd/jsbkzxrC117iwtQih70iw0jpU0JolHgdF2HogsG5TDY4/+7o+OZjnXXVP2MevWb8Fp116W8Tna2xqxSkX32LCShmGSTjnMdWcx10OUm6ci76AJJufwEMj/ip1oUgBiTURjqScx4DTZ2z1bigH5Mt4c5JPCYfuv8ShuF36YhmBWU5ocL5j7E/unzBjoL3jdx4TCAf7C2Ykc9r0CFTnUUarh8S2dbZ0sS5sPaJEwpThNjqPBertjubk/l6pfRC1MWTxyDAMMxDC1qniPJZWAyMmAj4vHIu/suTdXoSFKUSrz+dbr7WlEWFrM8RIKDOzZbw+0YdcJ7B7LnB1lYzRGeHzMM0KW5dEyXeMhhnjCROdLiOKbIwW+4FimcB9X7cCHzQD37dZm89QlBu8vdk7ibnW8TmPLgOdx+1JOo+9yt+fbNqUGRaPDMMwyZCVC6TrBvTmF0NO166MKeA6YtUCSO3NQKGWTGUioqKXBKKeDSFha4/J4pF6Of5vsg/5LrV/IDmONwyX/VNLQp1Hs/oqFrvUJ26MU7CprXqMdR7Lk8h5DIxwNMN5DNxHs8aPWG59/n6xv6+jrAjJdFecrXo8xoWtK7Q/09oEK61D15Su/JEZH3WI+9SkKSdPP3AzFnz4lDIa7xcH7tXv78yeNQ0fvHg/1n//Or556zH8WhutxzAMM2hC1iTQOlS7QC4aZu+aXOnAtJ+pX//4qWWbLdXEUmhxiOo8qqKRXDgzxIiA8hbfmexT+kou7ABm/eTAx81q4cqYkOkyArcm1swNW8eOGWI20dGEhBk5quN1IyztRlRbf7w4WGTFHLb2pWDOo9ZXNGUKZrKzMrF01XrccM+jMT1+RNUwPPuPW/HNDz/hkJMuwxPPv4W/3PJbpbKYYRhm0ISsW+qBhm3Kl7LdeY9VY4HMbKC1EdhgTaEMIdrSiOpZwdLOQPseqnIWYs0M53GPXLVROQnYI5c70OqVcOl6B7p1BStimknf6mazxGN8LzQwntA4x0i0R4p3NKGyHoPFY7puhKXeebQLUV39waLgqiZqGm552LrQmJzHXm1NKRO2pmpfUfEbC7/51S+waesO3H7fv/2FIHvsMgUXnH6MMjWFYRhmUFRatzQAPZ3KlBm5uAKoWWFvviOxg0aaWtfpVoSt60IECjmPhy51YEuvuRNLiHFaOHRBB4XPJX8T6nu2SPjDSBlNHqA5pL+iWTmYxdr+iLcpt1FNwkmkqa9NSq5Vj8Fin3JPqSk7tXBKJAfTrIKZFTUyNtXLGFkqch5hebV1uT/n0aCwdaqIx3jZbfokfPXdwqD7aNzeH64+P+LvOCUpYgNshyTFlK4rUZt2cWtBlSGvJ/n10H80FSgVEOvg9YSH1xPAU1gG8iscbQ2QOltB1xGppMq29ShrKh+hrqmhBi6n07L9o7aDkRVxlh6yra+1C3G6E/AqLXG8yHT0fVyyTMxW17C+xxH03A/scKA03YtFHX236VHeA2VkO+lnxinaMv/+CF5Lf3gUwe9FpjPx/VPglLFouhudPuDOLU6Ualf7Jq8T6aTc4kAUPpMza8TxGplJZ6dP+TCR7oxfhhh5PpO4ys1UX2BbpxML1gEjtWBCRzftqxieRFbPGWruncz+ob6X5fnq141tMW47DMp0Pc3tzk6ncy++3+/16hqz2iUey0qLUNcQPG2Bvs/Py0FmRjq6e/o2c8rLzERRdnbY58twuVCQm4POrti8bleKTWHh9UROhyDK8/KQSvB6AvSUjYSzsxUu3fQU3j/AjpIKUFS2yN0JV3cbaHios7TK1v2zbdhI0DtkcVcz8nSFMmavZ3g2JWr1wJOWjerC8O/hRGY67TE3ctNcqC7UepMYxNQ8dQ31Ut81/F2b2FEdUjskO+mc7kVpdiaqC7Xh3AZQnaU+r5yeHdfzZmbQ0XMj1+VEdYKFTrtl9qI8Tf1bfWKcKgZoDGNGbiGq46yYzs6kJNFeZCexHj1T8uj1taJZTkvq+Yw4n0vySMi2Kw258zMLsGYbWaHd6HEDJTmxra0gS30Och6TeT20FqdDXUumswDVhYlbvb1e9dNaRUEOqgs12zlG1jc0DMxq67bubnT2hu8Q+vLbn+Gi3xyNB598Ax1RBCQ5WSSMPD4f5BRx1ng94cnJysTFZx6Dzz+fh9q2NqUXpt3QJzd6Y+L1qMi5RXAfczmk+q1Ie+oW29cTip3rcWer4qdlxxZIWs5jb36ZkttX19Zq+Xror9ldqBbsNG9ajdbmZsv2T06FGoNc09qFrc2Ruzw3pqvvOT6vG1ubjR3lWFWtruHH5u6oa9DTXq7G+Lp76HeMm+WWU6WuZa2yP2J/3jrxmd7nSXj/zCpS8/fq3Gq4udClVsFvbo6/EqNefOHzGnK80jLVc3BjZ2LPZ+T5nKt8vnAo+Y1bmlrw3kLgd0dJ2FSPmNfm1cpH0lyx/044ihQt7EB9q4zNTYlXzKjOo2o3tvd2YKsJ01JNF4919U0oKwlW4vR9a1tHWNeR8MoyvBFOiLWbt+HF/36Kc089QgltR5I9pNcz0lzocasBALvh9UReh0+W8cpbn8HX2au8EcRimVsFr0cjtwigv7ei8qDt8/6hghk159HTVAs07gA8dLVOhye3EO7mJuvXk52vTpWRffDUbgEsPF6iNc2OXjnqdrpEjhiiPy5+ZKVND7Gi04destpioFtbghP0O8atpsi/P2JfC9Gp5WRStXWi+6fIqYrHb9qAC9Y6cH65rDThTuT5xHrSJGOOV5m2tm39nCf9YcT5nJtFr82BxnZ1X6/YBpx4n4QdLbS22J6jS5EyqoD0yt6ExwqW5KlroXzHZF+X26uKR6eDnivBBdkpHuf/tAIH7TMr6L799tpFuT9RNm7dgQf+9VrUx1DeAdnH9CkgFS5uvJ7Y1sOkKJlayC2NMt3pbSMVPgKlALQ/qM+jqLaWfUDTDqBsuOr+bVlv/ZrKtGKZ5jrA02vtprUrCrldsbR+0drpGbp96u1Is331IxH7w6zWQaLaOp651kaNJxR9Ham6utEj4Y81iT+Z0a16hqWH7wdqZ7GMviH4d2vie39z6y6hFLpOVKuV+6fLIGl6TS6YSahVz9SJY5R/xIjqYcrX1RVlyvfX//Y3eOCOK/2Pf+Y/72PU8ArcdMVZGD96OM789eE46pB98M/n/mvk62AYxkzSdflaGZFz2SylalzwWEA7ENvv6gB6NbWiha7dheX2rKl0uHpbt9XiDcsRW/WE0itrzprBKdeid+DmXrXhdNytcQwUj07ISqg4mT6PybTqCVS+I2mMHt9YqfWcpIbxqdKmJ9bK6mjV1slWXA/TchzJ9UwWs/s8xv20M6aOx2tPBOYx/+Hq85Tbl9/6BFfecj/Ky4pRXakKSWJzzQ6c8ds/KI8799SjsW1HPa6+/R/cpodhBhIZOvFI/QN7kninNYK8YuA3N6tC7fHr7W/T0+rPCgMa7RaPmvNYv8XSzRY4A05if06bWRNmRJse/ci7WDBj1naxdnUlF5TaA8W1HgPEbLm2fSNa4RjdWkk4jzTn3G6KtOkyFLZOFLdOPCbT69GoBuH6NaVMn8c585agauZREX9OAjLc7xx68hXxr45hhirUQumX5wDbNwALPkkt5zEzB2ips3M1QGEZIDlUoZSWriTy29sgXFedKJzHApvEowhb11vrPAqni3r39ef6mRUmHp+V2NQSM9Yj5lo3e6kpjZTYhJkkxNowrSjJCPHo1o6nUWK/Qts3YlxkqoWt40WW1VA1dXlKxnmsEGFrA5zHlAtbMwxjAeUjgRn7A/udgJSA3MZwX9sFFYQI7BwFmB/Oedyu3LjtCqkL59HisLXoIxhLmFQ4a8Y7j0jIeTQixzDidJkExJt/trWUfP5pIhNlzBTXLkmOOAPdzrB1Ms6jUVNm/GFrA3Ie/WHr+Lr0xAyLx6GCwwmccAUwO7JrzKQQQhxRMYbLpL/+eEjXrsrCebQbUaRCFFXYtw4hEKlYRtCtzuKTab601eQUqPvG5/M7oFYh8h1jye8LFGAYW3g1Xgtbx+s8ivUYmYMpwtbxTpfRi7VkxiUGCmaQNEYWzIhJN5RHmci+Mc15bE/uXPQaMN9aFMwkO11GH7Zm55FJjrLhwE67AnsebvdKmFjQC7TcwtTKeUyFgpksnfNIowBTYTShwKterWU7RD/9nRPNtf51WEWp1pYmlspiM8RaUs6jFpY1smCmRNsf8RbL6MVjprJ/5ITcvRIjxaN/1nZi6wkXsqYxidQLNVVyHpMpmDHCeaSQd1mekdXW2jnN4pExxDnKylFdSGYAicci2I5eMKaa82ineBQ5j/qwNfV5VKaWuMxvaET5nuffCxx/WSAHlKiztlhG7zyGzrWOKh4lY3sqCrdvXbwFMyaE0QNObPxPKsLoia5JpBBQa0kj3D0hZgmnUfmOKRCyJopyDQpb+7S80AR3UFk+tdKV4PHKaEhyLcGzrc0R6Ck5YYYxOexITYS7tDldTGqSas5jUMFMiuU82iUe6UNYXmHfsLUmHhUxR48xs4/qsNEAjUKkf/sep/5t21AsQ4g8tlicNiFGjLyuiTY9W3uBTjGMOe5qa+NzHhsTyXnUiTVyQ4XYjjdkXWeQu6ffPu2jLl/yhTw7UqBYJth5lG11Hiu0fMe6VrUAJ9XD1iweh6J4zGHxmPKQQ5xK4jEjhXMe7RKPVCxDAtFNaqWtr3gkKHRNPzeLXN1c6L2PCqzDBvFYEk/BjOjzKNnfpkddj3qbIRnfqieRsLXeeaRQcbvPvnzHUOeRjllXEs9V6S+WsT9kTaHiQgP6POrFY5qTni9+9Vehvc1vazImXuEWU4o4bM0YKh6Z1CbVnMeUy3nUicfsPMh2CFpqUh5OqOlzDc3Oe8zRzg0qkCEhSwUzNoWty7TGz/01CNfn0KWZ4DzGWyyjD1sb26on8ZxHau3jSaJoRhwLIxqEhzqPyR6zQI9H2E6h7m2jOWnxKPnnWydCpeY8bjNoDrVo1cM5j4yxYWsmMjSXeN/j0TFmhn1r0IuhPM55jCoe6XO+He16Rk5SbzeFGbUq3EezK66F8/jTl0DtJvVr6nmptQuyp1WPPTmP/mKZBGwxI/oqRh5NmNiLDLihiVc01yaQbxkeybAip4qUmi6j3jZ3yAmPFBR4kqy29juPzQY5jyLnkVv1MEnP4BWweIwMFaecdiN8s49Cw+xj7FtHqjmPqZbzKKqt25vtE4+jJqu3m5ZHEY8WOY80x/qNh9T9sWYR4LW+B0pJAjmP6sQS2dY2Pfr1GClmk2nVk2xo3+iwtZFTgQIFM9KgGE1oVM5jZZG6lu0GOY9mh60553FIhq11eVJMgGGjgBOvBPKLlW/lNN0+G8o5jxQOTc9IHfFI6xH7Z8tqYNLukCnvsSaMiDMLcoNpmxQu3ryy7889ZKtkQ3amWeM8drSoYxEfutK2aTuiKXU8rXqEGIm3IMTINj3KerQCG1OahCclHuUEw9bqba2B7h5NmcmCnPQ+qhBh697B0+PRCOex0uCcR54wwxiD/uLPOY99IUF06nWqcGxt8LdasW89KSQe9fmOqRC21m+/Zq09RTPCddy+Hujpss95FOeG5sDaJRyp2XeBCFvH4TwaJdgq02S/27a2J/GwdTJNuYORDRCPia+pXAsN1xpoQBuTaiD7ncfUmC4DA51HyRDncVsTDEFMmMkwqVUPi8ehgt5F07c5YVTKR6mipK0RePGP/ibPpvfpi0SmLqcvMwcy9fNLFfGYkW1vc19x/nZ1AA019oStR05RbzdGcDu9VoWtC4LFo00IoUShzeY4JswkGwYtdMr4wwgfls5UbZ8tPTRbO/GwtVEFPHlOEZJP1nlMLOcx0KrHuL9Tt/Y3n8w+ov2S7UydghmjejwSoiNXsjmP243KeeRWPYwhcNg6OlqoGvXbgi/E5D6a2acvHLRN4RRTWNRBVbT0ztJrb74jjd0jh5bWo3ey7SqW6WrzF4aQeJRTJd+RIuset7oeM8PWSnV1fiBsbSP+Ho8x9hUU+XP+AowE/sQKnLIiGkX17sIO4KoNifkhojVOhiOZs0jGzcNlzMiRUaQJiE4v0B1nz0kjnMfAXGsYhhEFM8J1bPXE34vT1LB1kj0e9WHrRJzHwmwgK93YnEezw9YsHocKXG0dnTxNPLY1BBcbKBf/BOJgRoRlZR/QUgeQMKLctvY62Oo8drYCLpdaQays0WuzeGxXC0UoVJueCS+d183N1syzLixTt0s5l+GwImxNDiwJSPqAQcfGRkSldSxtelQkpT0OCZFEw6CTs9S2Ly0e4Ny1Dvy3MfGG2EaEZGdkA7eOCBYhy5NoiJi48yibVDCTvPNYmZ5i02UMLJhxi1Y9zsRdx8Z2GT0egwtmuNqaMTLn0bZwbKo7j21NweLRjvnEQjyS00frocuBnSMKhXik3D5aE63Hzl6PevFIAo4EJL1ZFpRZ26Jn23qgN0J1hhXzrUW+Y6dBIyni5DdlPuybp263VPR4dFsXBhXj/1Z1A282SkmlUhhRbT1Ke4td0w38ZrWEE1c6cMTyxC+x/mrrOJ8ixxEIDZviPCaxj4Zp50kqhKz1zqMRYWtPEmFro/Md9a16zOrzyM7jUMx5pLBoaB7bUEc4j1qxjDIVhPIMbRWPHUB7k/1FM3rxSC2faC0UvnbrpqrYkvOoveNT6Lq4Au6Ccmu2P0rkOy6L/BjhPJoZtg4tlrGQ3XJk/Hu8jCaPjKp5jgScx+TFSLFLa8JtgBDpEdXWSdgp1drYvcUdwAv1yfsyiU69Ea5jh9fY0DBVWxsVtt7ea3/I2sjRhITIbnIlMGHG6HxHolcrmOFqayY5QnPUOO8xQti60dqCh3Bk6cWjmssm2ykeRc5jT6f6j0gF51GM4mvaYbHzODl6sYxVYWsbi2X2yVcvckUu4ID8gAsYT0PsZMVIoJo5eSFihPM4XHuL3WKQMBL7J96cR3+DcIPdPSP6PIo2PalQaR0UtjbCefQZ4TwaJx65YIYxPueRxAi5N+0JNEQb7GHr1sagi78t7XqE80jVxG0p5DxSiJYELe0XO9v1ZIVxHumQUR6iWTicwE67AbseDBSUqKkNkfIdySfyagUzVoStbSiWma2Fq4mjimW/zxKP85isGPELVgNyxMR4wkQqmwXVmjCqMaiuLdEJM6J4yajRhIH1JJ/zGHAekWJha9nWVj0VBo8mDC6YkSBJxme2sHgcamFrctbIZaPiArsKMFINusCLUKhwHq0IO/bnrCnOozZBRUwSsTtsTSLK7kbh+mprnXgk51Ey64PXGTcD5SPU76lA5es3tEbgdoat7XMe99J1kjq6SMacNin+nMekw9bqbaMBIsnvPPon3kgJh623GCYeE3Mey03KK/Toenomsn+ICm0fpYLzSIKq0Mg+j74knEeDRxPq+zwS6U4YVogjYPE4FKC/EhG2puKCvGLIXHHdN2Td2+MvCPEXzdiS85jdVzymgvNIIWs6lxTnMRXEo+Y8avvIm5FjzhvauBmqcKTjMe9DYOHnAUc4EsK5TjMzbG2P8zg8XVZCtOQcdvvUcO0BWoubWBqEG1UwUyJyHg24KIom4ULM6puYx+s8bjUobJ24eFRvaw0e/+dPM0iqYEa93ZECownzswCnIzXGE1aaUTCja4ZBoWujxSPnPA61udZaZarMU2bCt+nR9emzTzzqxFFKFMxk96m2tjXnMTsk51EINbOO1eip6u2iL4Gv3uhfOOpzZgdhwYwIWVNfxfebQ/s8xpPziKRyHkXYusHAsLWynoR0jYzhfvGY/HpSMWwtjpdofj7Qw9YiZN3eLftDvEaErdOSaNVjZMFMkHg04W2IxeNQynekcFtLvfo1O48B8rU2OK06UWBnwUwY55HcPx/1V7Tz/Okl8ah9RLc15zHEedTCx6blp46Zpt5uWBr777gHb8HMXlqGx9w2CW81BiubeARLsk6WCFvHI1hjGpeYwFWxUDc5xTjxmKzzCEPxJHm8nLr+k6kQtq7S3L5ag4x7b4JNwjPTAoU7RjqPsiyhV0ssNqNdD4vHoYD+4q+FuJSCGUYlr6SP8xgomLEx55EKZqhIRZudrDTBtjVsTWuxu8+jFOzMEsIldjghi5xMo6Cxh9QUnLaxeUXsq7Tiw4dNBTN7a87j3Hbg3SYpaFpMIgUziY7eTXZ2tB4fAq8jkaIZUWlNOZ+JTpSJ5IbGK9bMmGttRI4qOaIUJfbKxruiiTCmXH0h62uNHQeYFudbUKXmXXR0y2gzuIbVzCkzLB6HUtiacvrEJIpsbtUTtkF4n4IHG6uthcuniQOPXe2V9DmPfufRJvEoxiOGcR5NEWvCddy6Wu39GStmF8zQMRHzzrV2TlaQ6ZAxUzv0VCTT7JXwhW7z8fV5TMbJkg0Vj+p6EnceRcjaqGKZZJqEB8LWBuc8JpmjOkN7W9vWq4p1uxmjtYU1Sjx6hHiM85JRaUKltYDFI2OM8+juBjpU8SjnsPMYsUG47WHrEPGoiVr7nMcUynkUjjmthabLEB4TJwKJfMf1cYSslTUZfP6QozrrkECDclEsQ8cjWtW3weyWo+a8kQDYqE3tfKtJvfi1xznHOZBDF//FO88ZEFVGicfExwECVVoVsVFtepJZj1l9HpPNUT2rTH0CmgaUCgjncV2tMc/n0c79eHNCRb6jkT0eBSwemeRI1zmPmnjknMcoDcKVsKMndcSjqCa2yy0Ok/NoW5/H0Abh6mrMcfpobrQQaxuWxPe7Ro4npL6WJ18LHHIGcNxvVSEp2vR02FMsM1fZ/erF8rUGSREqX7cm5mQl4jwK17HTC3QZFCY2xnk0Thj5C2biWI8Dsn/aj/HiMXHnkSrjjylWX9CTtaklHo1yHr1BE2bir7TebqLzmJFobkgUuFXPUEBMCKH8ORG2zswxPj9swBfMNFo7IaS/CTOU80hoFdceuyrk9X0etVY9tjmP/h6YISMh6HjRsTLyeFWNVcPkFB7fviG+3zVKzA4bBZxwuZp3Kc4Nmq0tHFjLi2XUC63o60hsd0sYv8ChtO2xKofO6JC1uh51rFwi6/G36dHcWLucR9ovlFfok+PruRkLIic0kf1zepmsiPJ57cCiTvvFI+2jUaUmha2dCTqPBlZa93EeudqaSSrn0a31MdRcNa+4EGOoNwjP7+M82tYknFw+IeqFQPI7jzaIR1qLOH+UsLUmaNMz7PnwEdZ51LfrSTc+ZL1hWfzjGYz48EHC9dTrVOFIjdDX/aTeP2EXm3o8ypit7f657cECgGYox5vH5knCyTJyuowRYevhGbKhldbqetTbjDjC+lXa6d/oAbwG5xUmPk5Sxjnl6mv4d4q4jlXFQEaahB63jK26t30jmoS74i6YMd955LA1k3zYkUJ82oXXm8l5j8gr6tsg3IregZEQ4WDKYxMCxM6wtXAdCV3lN+ETjradowkFIu/PyOPlF49x5jsaVW1dNkI9H+hDzVO3AfM/Vu+fsKuux6N14nFUBjAsXa0Cnt9ub5i4WGsQbsR0GSPWU2Vwg/BExez0HPVFLAv8mRq+f+KNgNI0oqnZaorBS/WpFbLeVK+6tEbgSXA8oXAeazjnkUnpnEdCy3tk5zFCmx47C2b0c60FWsGMLdXW/g8ePWqBCjlwmvvo1QtLu3o8mnW8yG2tHp9YvqNRzrUQiI071Ep3ckApekBO5NhploetR2oCaUMPTWRJXgQkU20dCFvbK9asqLaOJ+eRCpqIBSHOsBEInR7v8Tpbcx3/0yChVRNYdjPW4HzHZMYTimprIxuE9815NPypWTwOqbnW5BwRWt6jV7g4Q5lwDcLtDFuLfEcRHtYJJVucPiEQFddaQ3No7XEew4tHwycCVYxW2zRRU31tKlNcGOFch/ZxJHd1vSZky0dqP7NOPPpDxQa5ff5q6yTC1sbmPCbmPGY7ZBRpF2cjw9ZiPfGIx1m56i/NM2DcnhFpBjkOGSeVplbImhirtelZZ6R49CBu59HlAMq1bCQjG4QLerW/VXYemSRb9fSEiEd2HsONJrSsyXMsldZBDctd9rbpCRWPdhTNiEKRCDmPhon90mr1tm5LYr9vhJgNN35w1YLgx1gYti5Li39+dUzV1gmFrY0Xj8Lpi1fMimKZNi/Q6oVtTqhLkv29FOeb4TwmUDBD4epcp9ra6ZuQP9nUqLQ27jk9olVPHOKxvIDa1qqTYBoMSAUJRcyzZvHIJBl67A4KW/tYPAL5ImydIs6jmJ4STjzaUfmtbxAu6FHX5kulsLXYR6Jx9mAQj+HGD65dCMi6smYLw9ZlLmObT4vq3cRav6i3DW57C1SCQtbKZ3PjW/XEKq6nZAFZDqDFA6wxeFJJogUzYhyhGs5PHefR6DY9iTYJryoOFMvEW5MXV9g60c7uUWDxOKRyHoV4VN0KDlvrCmb0DcKDBJvFbp+Y3KLPefTqxu9R70ErSdeNJkypsHWo89hrjvPYUJPQrxviXPvD1jqBSI7rljWB7y2stjY+bJ1MzqNsfNg6wb6T1SY0CE/EedxVK5ZZ0EFlkeY5j2lS7CqnVDtOqTCOUOB0ACMMbtOTaM7jaG0dG+tMUI79FMzsN1nChMrEn5vF41CAw9YxFMyE9Guwq0m4v4+hTjzqx+JZvZ5oOY92hK37cR4N2z+lVept/dbEft/IgpnQ0PTqBYFzNHQ/mIh/7J3B4wATch79OY8mNOVOMGxtZIPwRApmdss1L2SdqNgXzmOtwaMSk2FECQk8CV29sqEjAT3+JuGx/86oMnW/bKyHpeJxeAnw4uUuvH+DC4dMT+zYsHgcSgUzwj3igpnoDcJtDVtnRwxbKxjZxzDeBuGC7hQIW3eaWDBD2xDCrT4x5zFYzErG5TwSK39Qq9+3rYOVlBnsIiXT59GMsHWifQyrM4wvlgkKW0v2F8sQ4jNDWgLiMZWcRxGy3lBnbKjYo1WSkzCNlVGlki3O44gSdbuZaRKeuNCJE/eM/4+QJ8wMpbA1zbYmOkSfx1wMyRkzU/dWL8xURRuuQbhejNiV8xjkKMnmTFCJK2ytz3lUv/ZaHbYmsSqKhsxs1VOiuY50fgi3Pl7EepQ1uYI/AMQCvU4hlEND07Sux68LdoMtDVtLtuT0mT1hRjh98Yath2tha+PFY+zOI4WSp2db5Dw64nerjR6VmGr5joQ3gSbho8rU20315orH0FY9hdq54vXJyjjFB852ISPNg+e/ls0Vj2eddDguPvN4lJUUYdmq9bjpj49h4ZLVER9/3mlH48xf/RJVFWVoam7F/z7+Fvf8/Wn0iDpyxtqCGe3WlgIMvYN2yu/VXMP3nwzkrJkNCYOjLwq+L7RBOGF7tXWY9dghHjN1oy3tdh6FU07HSy/OdKF9QybMJJnv2NctTotfPIpimUih6dAc3QEYtvYkmGOY6ZCR4zR+wkwirXHMahAebxh9Wra6bposs97AEYnJVluXiwr9FLrUj9Ha9BgtHj0JjCccqTmPGyx2Hgu1y8znS2Wsr/XhvIOduOkEJ178xhNz0/S4P/Mdfeg+uPWq83DfYy/isFOuUMTjCw/fjpKi8A2Mj/vl/rjhsjNx32MvYf/jL8FVf/iH8hzX/fY38W6aSZS0kCbh2oXXVvE4fIL6b8pewJm3Qi4aZs12y6oDhQcU9qPbxV/2fZxds639fR4jVBPb5jyG6/Nocc7juOnqbdOOvj8zUuyLc6QuwXxHghqq+3yJu9e5otLayvGD0ZBR6q+2tjfnUbiObp/aHscoEi3gCa62tsd53E0rllEn/5jrPMZzvIRbXWdgbmoiHL+HhNk7SUHO47odBotHX3wTZrIzqFVPYNKNGUSabV2Yo263sR34w6s+tHXJyM+SMEkLupjiPF5wxrF44fUP8PJ/P1G+//2dD+PgfXfHKcceggeffLXP42fNmIQfFi7HG+99oXy/paYWb77/JXbdeWK8m2aSFQAibK1zacz5vBMDZcMDX5dUwX3GLej6+Cmg+Qdzt1uoiVSaE/z2Y5EfZ5XApuc/8UqgcRvw4bPh+zwSdoXRw7XqscV5lIBZh6pfLvwsitg3wHksqU6uWEa7fFPFtezISEzQ+mdXW9eKJxoFzkC40jjxKMKgchIhaxMKZuKwVChcXOF3Hg1birYeLS9NWY8c9bWaXSyTqNgPFMzAVqfxH+eoJ82jH3kxvsL4Ho+EO86CmZFarWZTu4xWkzJQqH8knTfpITMlRdi6uVNWnMYF62XsP0XC7uMdWLZV1wrMKPGY5nJh+uTxePDfAZEoyzK++m4hdpseXgzOW7QCxx9xAGZOm6CEtkdWD8PB+8zCq++EuQBoOCUJTkdytTxpTmfQrd3YtR5ZkuDWch7TvG5ITidk2ecfNZWWngFJOJIW4ikfATpFHT9+CrlsOOThO6FhnxOQvnKBudstqVC321IHV5Rj4fD5lARxyZVm6jHzjZ4Kz5hpwJhpcLY2wqs5j2m93cqxErhJiCjaKAMOE9dDLT58s48EGrfDufIHuDOyle063T1watv1ubuVfUPiMcOi89k3fiY8xRWKcE1b9m3QvlHW7fWoxzUtPepxjYXeMvXjt6txW8L7ms4Zj9cDOS0DaemZfdbbH978ItC1SOpoMeT8S/b9pzKDzgKf4vTJDheSlejK/vGLNQnpcayrIoOOtA+Nnvh+r7/1CDGbGcd6Rir5jj70+IBW2Yn0OIol+luPELNEjiuwvnDsnqte8Bd10RqMr4NVj5cQ+7HuHxnlLlVRNXuN2zdiPfrbWEQacdEhgcdvaXAg3aC3rzSnE15NPNJzxrJ/xmk+BrmORp3H+vUQHq96LmSlBR+z4lz1WLR1qvcvWA/sPwXYY7wDL34toVe8GKPEY3FRPlwuJ+oaghsq1zc0Y/xonZOkgxzH4sJ8vPnkHyFBQlqaC0+/8i7+8a//RNxOXmYmirKNCYmV56VWRbHV6/GlZWCj9nVVViYc6U7IDic2aPeVFhbBqc9ns4itFaNAH9RL6zchfdVcbDnpBniz8k3fP9tKK0GvtqSnHbmFmrsTht7MdJDvJLnSUR3lccnSNHoShLfk3f9EpZcjUZnuglO33a0+n7K/CvILkG3iejpHTMaOfY9Xcu2qW7Zhe3auepzSnP7t9qa7lH3jS8s07Xi1j98VzvZmZG1XK4q37Xm4IlgLVsxFcQ69NwS/P7S4HKCSp4zMbJQnsX+oCGhTrlqBX+3thCOJ59qkuddlRcXI8MVnLTSVVCjnRa67G6UGHu9Ej9fkTHotTWj2Ogz7e+iV1fedvDRXXM85Ppd+rxXtiO/3+l+PmipSlJmB6sLY9tMBOfTBuwWb3E5UFWqdGwxbT0A9jiksQIccXhSmSzKmZqsjNLc6C1BdaM4HOrfyTgBku2I7B/IdPqQ51Hhsem4hqg2Yh57I+TyunM7dLtS1SshMk5GXBXT2kAimfWXcmjya4CKXr7owfBqfnukj6NzpQV1rGqqFFWgwGS4yjnpQkBW8jcoCiiR5ADkb1YXp2LCd3l07sdd4OrZ5WN/QYH+19exZ0/Dbc3+FG+5+FAsWr8ToEZW449oLsOP8Rtz/+Mthf6etuxudvb1JK286sWrb2uCOQUWbjV3rkUXivc+HmoY6JfChvCXRlArJgbquHnjbrA2NkbvlLqxQvm7auNKfi+lzpZm+f3pz1XGETTUb0NIc+XU7HWqRkc/hxNYoj0sWd1FlYOpPjlb5TWKpdhsk3SQRjybwm3s9aDJxPZ69p6pfOF3YOnomvFoYuKGxzr9dWXb5Q/pmHC+5uALug85Ql/Hla5DWLYKneoKSR9g55110hTtfO9QLf7eMpI6Xr1rr2tvagG11YXIr4/h7l7ReobVd3XDEuSaP8qYPdDbWGnL+Jfv+M7NQPRe398qGrcedq8Y0fV5PXM8ppavr39Yd3+/1u55iVZx53D3Y2hzbPpqarx7jz5uN2S/69QzLC/ThbWhridjTcvccnxJKptDwvHpqw2a8SFP2T6b6nij5vDG91uxMVfzSxJv1Tcbm7sZzPqtnrgML1vlw5+sybj5BwoJ1dLxaDF1PZZEaNXI6YjsXCpS8Qwkra9yGX2PE/mloV6+tXjl4G+lKYq+EDfWd2NrciaalaquhiiLA67czDBSPjU2t8Hi8SpW1ntKSQtTVh5/qfe0lp+O1dz7DC298qHy/Ys1GZGdl4s83/x8eeOIVJewdileW4TXogkQnViwWrFVYvh6RI9fbHfxHRjliaRnwOJzWi+vCcrV9kKcX7vptgbw6p0sJ1Zi2HtoX2kQZD203ynYobKzgSjP3eFWMUW/ffhQ4+FQ1F7S7E+6Q6lxJq0an42XU30bY/TN+F/+33l0O9OcQemjijdhut+qiUUiW9o3hx6ugLLCG/U4I5Dqu+B5ualMTbuniA4jTldz+EWK+bkvSx13yePzHLNq5FpZs9UOft63R0OOd6PtPoVO9BNe6ZcP+Hjxa8NuF+J6zwOHz514a+bfp1pw9Zxzr2SdPXcunLcbtl6AP2T4gzQFIMh238KJwVo66hrlttD9iy1dLNucxltdaqOWykqg16z00lvM5P1s9rg3tMtbVenH2I6YsBfom4bG83uGl5BBLypp6veZUH3TRCQQHXM7g87MgS5V+9e0+Zdu9ncCyLS5MHwXMHO3DphiaOcSVHOH2ePDT8jXYZw+t6pFeuiRhnz1mYP5PK8P+TlZmBnyi6lBDfE+/y1g1XaY7NQow9MUy1ICZ3DWrJqgUliluq1I5HDreLhQxYYb67Zl1npKQpX6TVJm7eRXw+j/UGdsblthT/T12Z7VJOTVMp8bUecWBXoP6amt9WyUzmpZrYWNlHXQcsrXQ1PcfRP4dsaZk1yMqrRNtDm7UiEJ/tXVqFMwYPdc6SIw4Ep0uA1snuhQ6ZczQIoFftprzHtETQ7ue2Xnqg+a0mXs9jbfPo7+1k81teoq1t7CmDnPLQz0i5zGkOKXfBuEmVVoTohtinz6PWl1ms64u84e1qi6bNS629ccdtv7ns2/i/juuxKJla/DjklU4/7RjFCfxpf9+rPz8gTuuxPbaBtzzj2eU7z/68ntccPqxWLJiHRYsXoUxIytxzSWnKfeHikrGzB6PPanR+kUvHuu2qLd0kdXC6OrF36QRCUVag6+mGMrs9M4fCWwz+lBWjg20hKHnp4rrh3+nislI6zHzeE3ZU71d8b3agHqf4wI/0zek1jfOTksPFpZGzhtf8yOwbC5w+LlAzdqoE1UMmzCT7FjCcOLRmZ5EtXVqtOopM3iuNeFJslWPkdNlgvsYxiYy9skHHBKwvBPYYdL4vf4rwGXM1j5bmS0e4z1eosdjrcEiP16KtLY0TSZdVkJb9RBUsxTNBKbzhkYEmjldJnqrHvW2WSeo562Vce5BwO7j6GTzGS8e3/rwa6Wn4zUXn4ay0iIsXbkOp11yK+ob1U/I1ZVl8OlC0ZTXSKHpay89HRXlJUrom4TjvQ8+G++mmWSmy4QWxRjZ2iRZ8Ui43epaSYyYheglGa5PYL9Nns0Qj1rIWi+KwglHTRzJZh4vel4Rsl4+V3X9Zh8VmOiiF4iyfuKNCeshx5MgF3bzSuCxa/v/HaP6PIoG4UaIRy1sHf+apED+a4o4j4F+ffZPdCnRxiSa5TzGup4D8mVTXUdlTb7ozuPIdHW2NkUnzRpLmGgfzMBoQnsjjFoqotISxwrnUTQKjyYeK4tUh7LHLWO7iX/i4ZqE09pyM9Vj0qzrwPbDWnX/TB0R23MnVDDz5MvvKP/CceJ5NwR97/X6lAbh9G9IQS7aMRerF6Gv37R/rnXImDXKoVPFSAo4jwSJs/QMYyaERMu1JJr7dx6VYhUScpSvZtY+Es7jtvX9P1aIV7PSDMbPUF1qyims0cTsih+AqbPVDx6hucl0PtF+EQ3ozXAeQ0ZGRsUIJz0jOyBck5kuk2zYOjtXFe10DlIhVQpg9Fzr+MOg4vyT/EI2UgGJVWHr/TTx+IWJh8gftnZED1n/2AF065wvM/C3d5MGVthaiMdGk8W1VycWlV6PUV736DJ1J25pQMwTXZITj4GDVqClWvh8wf0ltzUDmxtk/9zr/jC+IRSjUjEKmLwnQG1PxGSMVBhNaPf4PbooUr++Ps6jJm7NECN9wtaxVdGKalnT9pEoltkei3g0Oc2AzlVi+XeB++Z9oAqYcGJbNJo3wyn2i8fwRXhhMSKHV7iONPrPgFB8wucP5cESne0RnWirMXqudTxh0DynjNW7+PDuZJ8SUi42PWzd/2MLnDJmaqLETOeR+kdGcx6tCln3FftyzHmyds+1LtJ6GjaFmfJppvMYjZFaU4cNJs20jjbbWoSsW7r6egLCfYwFFo9mIQoNiMPOUipTUypsTWFiO8RjcaUqIGlKid5Z8hc8pFkQtq6NM2ctzZy1UENwEj21m/t/vJlin87NcTP7ikdyIJ+9E3jtgb6/4xFiP1XEowEFM8NGqrc7NsEIEj5/Umy6jBlzreMRa3vkAmMygUMLgftGyyETZuxxHvfV8h1XdAHbTQzL9vTjzu6tOY/fWiIeA1/H4j76cx5tFo9CLJldMOOTJXg1G7E/8ThKcx43WSQe9WFrMZpQXyyjz3uMFRaPZpGpE48FpfDqiw/sCFtHcB4tL5jxh6yDc8okUXFtVs4jpREUlMYctjbdeRT5jrWbYnOXzBT7FaPV/U6O2w7RUl5j6xo1lB1C4HgZ/KGInk+MaIwjbG1IwcywUYFjYgCJO4+pNtdaX21t3HO6tX6E/YnH8Vq/QOKiChmFJonHeMbv7S9C1i3mirZoOY85DhkztD+VuSa7aoSYMBPrPipLgZxHEvgFWic4swtmQtv1REOErTeG7zxmGF29gTnagiLdaMJQPl/qw12vxxbtYPFoFtqYOeWCTJ9KdjsEPaUxZqKa4TyG5DzaVjDjF4+bIzhHJjm0+cWq40nboWIQ28WjyHdcF+NaTBSPwpGNp0hEnE9GHy/hOlLYOJ7JR4aIR5Ocx0TD1ilSLJPtkJGtXQyNzXmMTYiM1z7/btfVrJHB02RWwYwj9nzHL01OSRU5j+HWRI4sTfzb2ENzta11HmMJ7fsLZmystqb8PocjstNmNG5vfGHrjSY7j7WtgXGEInQdrk2PgNoGPfxhbF1wWDyahXBP1i4Cls6hMxgNs4+xfh3p2seukAuxaDpteZ/HcMUyZufQ6QVSMzXVkuOrljVjH/krrWPIdzRb7Isc1Mbtsf+OWccrkZB1H/EoJeZMi3OzNsR9tbraOsXEo3CQuqh+x2d9wcw4zXm8a4uEt7TPfY0eaiYimSKOovVUJPItyncMznlUF3dKqc9f5W1Vf0eBXgP2J/gdCKQX2Bm2Fj0eWzrlqNXPVjuPo0SPRxPb9AiB2KX1e6oo7D9sHQ8sHs3OeaTJHHP/p3zpFgLGlpzHSM5jiohHs3MeRbFMjCFrU90+EirDRqtfi8rm/tD2j2yGkPWLxzjG8XlMClvr2/TEgzhWiR6vkkpVmJPj2WRMLCnh80eMFE2VHo9BIWvrC2aE87i6W8JZaxx4pV7CPVuNF0yxhtFJOJLjt74b2GZySFbf53FKloxnJ8j4eKoPD4zx4YACIR5hEZI/jN6f4CfhSIYfOcRGFzYlVCxjgeuoF4/RnEdyQ4WA22Rig3DBNu2ttLJI3WZhlLB1PJg+2xpDXTxSYYjm+ply4Y8557HL/mprqvymKS9hQqSUQycbLUZ23kfN16NegYVx9Hg0O2w9fIIq6kmoUGPwWBBiPy0tJZxHyd2jHa90+9v0EKFTiuLtyylC1krxkjFugP/8cQ4O59HolivCeSQhRk5VOCeR7h+nvYWt6QZavRJOXW1SQ27h8sXohK6KI6vCiGrrnbMD5+WlFYGvrXIehZhNj0Fgi5A15aV6TZi1HXePR5OLZULD1tGcx+Ha5+O6Vtmfk2gm25pkjB0moaqo/7B1PLB4NDts3dUeCPG50gy6LBmQ8+i2oWBGFCSQMKD9YsZ4OUFJFXDkBep4uxfujW+6TJ9qWYP/TH52rHpL01PCzHYPi1mV3zR6UeybWIWsXqy5UiNsnXRfzvJRhoasCYc4ZvEK7BQTj6Va1Wy9SQUqwsnqDhNWpAbYJOZI2G0OeQuzy3mc4Bez5ositdpaVvbBRO00+r4NGJUBDEsHOrzAT7pGz2YTa55qoEE4UqRBuNXOoxTxQ6hwHRstWlNNkPMooyDbmLA1i0fTw9btAdeIwpUkREwaEp/yfR5HTFRvN682P4dOCCLa38f/NnAM7HYeh+8EjJmqitpv34p9LWZVW+eXqAKQ9o9W3BUT/uNlUti6Pc6wtZjCk07iMd32YhmxnsHQqseMudZ6sSbESHcUobaux3wHK9aCGVH9vcbgqZz9NQmfqKWvv9Yo4dk6CbeNkPFDe3AVtNmIMHp/+6hM+8Bhu3i0OGztjsF5DDca0Ey2Navb6es8ctg6tZ1HClvrQ2jKhcTkj9Bhw9ahOY8mOUfRGCnE44ooOXTpxjpY+hyyuHMeTRCP1DSeWPRlfGLNL/bTTSokqo3dBTXjeIUetxgr4kOPV8JTk8wQj4mcP/RhT0QL2gfvXGt9zmM0t0+EiNdaECKOtWAmEEY3X7TpW/XslKUucGWXhFq3hEvWWR8OjjVPVZwztE47sStsnR5FWYkJLy0WOcY1jSE5j0I8Jrl9Lpgxu1UPOY90EaGwmlk5awk0CU/YFUkUCidWT1C/phzESH0DjRJHwsGiavdOLaPcR9NS6hIoeEg3znkdPUU9H+a8Hd/vmlVQlEiltZbzaI54TMJ5TPR45RYB2flq2Ls+pJDL6oIZEbKmfNjQVBO751obLB4px9HbjxgZn2WhUItpgoqsK+CxznnMJOdR2+5KCxzPSPgLZmIMW9fa2KaHKNJCxFaHrV3RnEctbGyVeAw4j6JghqutUxgp0CScnEc7+yqKsLW727oCjEj5jrQWEtMhDcLV9RgsHkkQEFtWA288qL5e6qkYx7i3hAse+nUdv4jPddSPJ3Smhng0ZZwkfcDIyU/ceUy0g4BwHRu2Bf4u7HIeRTeCOHJzLZtr7TExDCrZHyIWa4m2nqp0IMepOnAbeqwrmCG3k3ptknhbb+Nnit44nUfbw9Z+5zG1qq2NqHaOlZpGdTuV/lY9xrixHLY2g4wspa+jv1UPQRclutDaJR4jtOqxrALcn++4KvyneqMnzOgLLzYtBx65Ku55xYZMLREcdDIwarK63+PIdTRd7CfSpkdZjygCM/B8pvQCygsm0SXc4jhIOM3AH7I2rlgm4T6PYt55jM3jrcDMSSEkhrIckXPoRIh4rQXOo6j+FuJRLyZDczBJOFqRayhC6aLSmsL3VuY49l2PWnTRv/OYKjmP6m1Te+pUWxca1GcxVrZpqdOl+RIy0wITd5INW7N4NDNkTYJNhK78F1srw9aSTjx22dvnMVq+o7IegwtmQlu+JFC5GnAeXclVM//ibGDmAer3n7wQfw9DM5u6+8VjHJXW+g8jRjqPNAXIf6zkxKub4z2nRaW1gfmOCc+2Fs3jt8fYPN7CsLXROY/9Ve9KujY9VoSIQ6u/4Yucg2mFE0r0+NQdI4plVlqwH2KaR95PzLLclSo5j6nX57HQ7zxas6YmrVF4VrqEiVWSf+JOC4etUxB/yLrd3qbceiFmZ8EMCajhEyPmOwaPuzM4bJ2AUDO0YOaoi1ThSPmW/3scWPBJYs9jxvlDYeKCssRyHs04f/zHLP6QtSFha4NmWid1/vidx9QRj/5qa4+JYkQK36aHXEm3D9hkQaiWcjA9/RTNTNA1LLcCkfOodH6h3pJd9ooxcbxckj15sv2x72QJ47XPw/oJM40WFcx4YnAeAwUz1jXuE43Cp2hZMW1dMjxJTtxh8WhqsUxHmAubhWFr4TpSsU5I02RTZyWHy+OifUJFO9vDhwYN3T8kmsUxSKDwwjDxWDYCmDpbDcO++RCw+KuE1+J3sI10+qhhO6VXUDg/3mkmIs1AFGQZ6TwmKPgTKpihvxHhvprlPMZ6/lB7KTpv6W81dAKThZDj96dRPtw32oe982TkB02Ysa71iyhMoRw/qxpNixzDSM7aeAurv/XrEaywsVgmoYIZC8UjzYt+6XIXnr7UZVufR7eoi3UCTgfw4DlOnH+ww9awNVHTpJ63k6slw1xPDlub2qan3brxe/GMJlTWY6F4HDEpULwiqs5DMbJ6V1TskliNM8/RUIEtpulQLt3KH5AUos8jofQKNcAGEqIpjt6XocfL0JzHJN3ihMR++chAgU6XsXPe4l6PcB1pyk0chV1Gs38+8Lsq9WJzWaXsFw0tXmvD1v4QsYWhWhKzOVGcRyurv8V69NjtPAYKZuiL8GtJl2QU2jDXemy5up7RZRJK8oDuXmqZY0/Y2uUEZoyScNweDhwwRcbjn/hsa9UT7DwaE7Im2Hk0u0G4mQUG/UHtR4juMGeplTmYI/rJd9SvxwhnzYCQtSHOY4LTUkyZ3WxkpbVZfR4THU2YjNgXU48MLpZJqB1WZWqErGfnqQqhphfo0q55qoAzo2Am8lQXKye5xOKEKjmYGdblYIZzHu3OefTEkPMoCqzIhWu28DNQhVZNLESSKJbpdlszBpBwewITZqqLA43K9TmQVjcJD+88Jr9tdh5NFY86ee+fEGKheBSuSrjedaIS1Oxqa8p3FMUymyLkOxL+Po9ptouQpAoegtZRbMg6FESvUKpGpnMoCUfVT1HizqO/L6eRYfQkxXZCqQ8VQjxuSGibpjiPNovHvTTx+OetEp6uk3BooYylnZLlTaetbBAeril3uDY91C6HRNEGq8Sj7hpPLl6TJzWcx2hh633z1QdtVd4irFtvRWFgW1OHS35xZpXrqK+2JrFYVRxYDzmh25vVUHZ+lnGh43idRyND5iwezZ4uY2fYOloLErMLZkZPVfP9xs1Ue/fR9qJcFA0twDDI8Uuo1Uq4HL4Eehb2WYuYoEL7x3DnMc5KazMKnIwQj4k4xcJ5jJCLmwxBs9FJ9EdK2VAfDVSMToE2PTL20j77zm2X0OqV8GqDZIsYsXKSSyzr0bfpsS4HM7AdO5uDxzrb2gEZNw1XH/SvWmuFruhjKJzHZVusbRAeGrYWzqNePOZraQ9Wh62F8ygwwvVk8WhR2JouJAmPTksUf0huk7k9DEOZdQhwyBmB70lEz3knOPQaipFOlt/xS1I8+jyp4zyK2c2KeEy3P2wd1JdT7f2W8D6iop2W+uTD1vE6jyTqSqvND1sra0qLPjGmpFLtD0v5ydSs3CZ2ygRK0tRw9cIOC3PoHH1DxKJgxsqcR1WsyREKeOzJwRSssDnfMTCFJ3Kfx5NLZUzKAhrcwIPbrV3vsBDn8RN/sYx14WGPznms1jmPZXnqfhMha6p29iZZ7RwP20LEYxMXzAwg59HqsDU5HVTtG6kFiVniccb+AeG4+Gu1wpgag/dXAKB3ZinUHc+c5VCECEmi0toQgW2g82h4hTyJPrG+eBuEE56e4OdKZJQejeK78I/qBwaauCP+NhLoyZnQ/qEuACQg6UNevBN/4vnwIdYUbR9Vjg6I2KgOpTX5jvPbg5tmm+9kBRdg2BEi1q8nXNg6MJbQOlGkD1uvSiXnMYy4dupcx/tqJLR5rRWP+rA1teup0MbxWRq29umcR237RGkegkcDWug6EjUhl0IOWw+EudZ9CkIsch7JVaKLOlUch8tp8zeddiUv1gQUpv7l2erX370LfPpS/E4WQSIimbm+ealWMGOUeDSg76RoCXPEBQGhpv+Qk0j1d6LisXJswGnOL1FvScQlWEket9j3h6yNz3dU1kN/U/Ra6G+svzVV2NMcPNshK/OlezShuJd2kZvTZm01caiTtb+WN7e8y7oQcdB6wjmPWdY7j/qCmZUp4DwKcRTOeTy1TMZOWWpLp4csdh1DC2ZcTgl7jRfi0S7nEX5ougs5j/5KawsFrRCrolG48j2HrQdStXWSxRfx4m98vDm8MAyt3tWLt0RHyx1xvup4UiPseIQjERTiS1I8Jtls2hCnj86BJJ00w3MwhTP889PVNk5UdPP+U4mtBbKSp6qE0RUBmECbm9Iq9XbF98CPnwHV4yM3kY9LXMfo7oscQ5PEo/+8jkU82lBpXeSSsWYXn9I7cN8lDqVJtnAev7VIPEaq3j2+RP3BW03WihB/n8cozqMVoxL965FTp9I6mth3STJu1FzHv9ZIaNflalqBiwJtikADlm2RlZzHvXay3nn0aOIxP1utsu7jPIpKawsbhAtqGoFxWqaSEc4nt+oxdcJMuIIZi8LWwlWJNDUjVKwliwgBUv7cB88kLEaSbv9CLmpugbHOYyKCX+Q7UvNtI3oyGhG2JseRnGESjhuWAU/cAKxekPh6PEnmqZZUBUK1G5YC3/wX2LTCuuNlYpueuD400gcusRYLi2WmZAEFLmDPPODUUhkFTlm5j5hrbMvLfqub9QUYOQ4Zh2ku0usmFuuEXY9fzMqRRyVaGD4W+6fH4vB9vAUzNw9Xc1SpIvxhG1zHcu0tv9cj4+sV6k4TVc1WFsy4NfE4qjR4H5QqOY/U41Hrs2hx2JrY1hw4pzlsHQU5Mwe+ZGYSmzVhxsi+eIlWWscbUosnRNtcl3DxhL8gJJmiGXJAaewe5VjGOzXFSLFmcL5j0HqcCZ5DJFBIqJC79eIfEy9yEeshtzoziXNaFKvUG1Mg4vCL2bTYPmSUjzBfPMYy9YbWkaY5wYnknyZIie7t8ZYRstIGhsbeUli2zqKWMOEKMH5RqI4lpHX81JkaztqYjMCoxI0WjEoUrOyWsKxTTSOwMnwfT1h/r1wZ11WrP/jtOgc6LXYdiUot37G2BViyOaQ4xMKwtVv7/EqNyvWU5tsz1zpS3iOHrSORUwD3RX/GDupv+Nxd1m6bxp2ReLF7wkx55EprgdL6xTDxmHxlsepk5SQnsMU62luSzuNMKkxscKV10HoSzZsVTl/d5qSFI+HwuOFN2LmW1ApjoqEm6bXEnRNaXGmNYIslD5PaWhFKyN66C11JWmBbYzOBv4xWHZu5FoWsIzlZx2kh6zcU19Fi5zFCn0cS18T37dbmYHb5JExfFGVQssV4QmZbk0v81ASfMnv7uToJrzXaI3ArNO9ie7OshK31WBu2lpVzdkSp+n2PW0ZGmuR3Hu1oEB6u4prD1pEYNVkRcb10gbAr35Fy9vShYe1rSybMUBUr9Vb0+aLOyDV0nrQBYslhxHoMqrROulWPGeIx2ablfrFmjNPnD1snMt+anFn6oEWCL5HxiOHWE87lmzgrEBIOl++opHWY90YeUxHPGE08rl8CKynVrIM2LdQ2QQtZz7EoZK0Xj8Lpy5BkHFGkiUcbhIgoHNI7awcXyDi9TIZPBq7aMDgvmYlOBPrraDVcTW7sZevtc0aHFajbpl6Ka7arok1gR9g6jdQ0gKWakBXOo51h62DnMfnnG5x/CVXjrBu9F8t0maDq5jTrQtbU/Fls1+zWL/4wbeKizZDQvhCPRjTmTsZ5NGB/GH68ik0Sj66MxF3QplrD5jj3OV6U43n8Zeo/O/Idif6OGd0vxndaLR61JT1dK2GTLhRrpfMonD4h1g4pBPKcwOYe4AcLL/r+9YS06sl0yHhwjLpIqiCe12F/6Dg1ZlsDEzNlnDdMFdVnr3EoDeXtQlRab2+RFQG3WvcWZ2m1tS/4+582yuELZmx0HqnqutuAmeODUzxS1abm8slW54n4ezyGvPP5hVGahSHr6BdGRzIFISa0pTFkykyugc6juPBTE2uRimBTm56EqokjNQVvMDrHMD3xSuv6rTDNmS0s127LgEwt2Sh0LKGZldaxFMwM30k9nvRhx6Dwfbw5jzTD+k5tGge5kEssdEVCw9bHFQdcR8vfu/WtaLQr4/XVsuLI0qi9WzYPbeEY5BQ7gP20dkqftwJfttq7b0SPR3Ie9Y6fXdXWgkWaeCQnkvId/a16bHAe1+7QxkYadEkafDmPlMMnZjoLsWZQtWsyzqOhIeKYi2Ui5zsaXsRjSM6jWE+G7T0elfXozxtyiHq9toatA2H9tMRENU0wIZevudaQ9SR1vAzOdwwSj+J8zs7Xba8K2LrGeuexv2M2Zpp6u8Fa15EocakXk3qP6j4OTyfhaG1hhr5Ahdq9HKWJxzdtyp3r0a2nIk3GNVXqHZevd1je9DoV0R+v2ZqbZlVbp1hGE1LOIyHyHn0+2VKh5g65RKyvJaFI/R0lZUShv0m4xX0eiQ11wG8e9GBLozGu5+ATj3RR0L9RKyG1Lnuny1hdMBPjhTHpHDoBvabsvKRFm8NrwD4yaDRhH/Go7KPu1Ki2TmT/CLFmaJg4CedRhK2NFI8ibC3OZ8r71YfshXhUnMgcVdjVm+z2eWMUj+uXwmpE2LrerQrG2zX30Ur0zuOsHKDYResBvm6FLfjD1g61YTo5bIs6SMzas55Uw6ObCLSrxQ3lY8151DuPVBhCYXW7nMetjTLqW1XHkfpQFtjY55H4ZIlx23UN1pC1H6ta40SbLqOfMGO280hFCJTrFa3HYySnJtlQMRUJJTKtJFWdRyqkoDXRhd8Vx58KCRPxGgxYhyHutcHFMsFh64wkwtZGOo8hYpbaNoVuT//hiorJDBLSUdtPRTpm5IyKtdjiPKq3DRYGZqKNu5uZo35DuY7UsNwO9E3Cd9XWM6/d+qrvVEXkqJJLTeF8EmZW9QSNJedxh+Y8fr9GxmtzfVi0yVqR5ta9nXh9Mna0APVtMsZVUMW1va16jMY1WItl/FjVlDu0QXgf8WjSLOlwbT+ol19LA9DZZv7EEiPHARrRJNzoXENyHxXxmBa/+9nZGjzJx87xhEI8UhGVUetJVOxTaocIKRu5ntAcXr14FK/fypB1f3/3okXP9o39/q2a6TzaKR711bsztc/dC20sStHn9E3UxhEutCHEmKoIZ3ZX7TK3tAtosTmcn5cJ5GSqa9imOY9eH3DZU+Z+MOzPeSQXlNZRr/1pV5dI/vGAVo8nNAPHYBePsuXOY5jpMoQY/2e2mJ15gHq7bE6/DzUsbG1QiDbpHEzK6SPn1UjHL5Gxkvmi4ts41zFiKxobnUdJjJCM93iJkDU1lE92LKZ+PaFFafqwtdimvk2P3eJRtOixwXV0QlZCxASFie1CH7YWzuOPNopHtWl5sJi1cz2phjukz2NKhKw115FyC7uMeztJ2nncquUWkvNIjBsm+XtBtqXAtKBkGVzOIzkNlM8k+4COVrXfodXOY5jpMoa3xYlEQSkwdmf160Wf9/tww8LWBhWHOJJp/UIUDQvs+ygtiuIiEcFmQrFMzD0DLWrTk1RfTjFZxuDqYv/5LGnV8dk655Eqr6mYjtxJO5xH8eGD3pNoLeTWjtnZlhY9RJH2zk9hxyY7nUdNjOQ4ZUzTQnp2On0ibD0yQ0ZVurp/rJ5yk8q4NXEtsLInaH/TZUS+o514gsSjelun5e+O1y5PrRaWYJiJa1C6jvU1kHw+yPRGbbXz6J9rHSlsbeJ6pu+nXjgp+Z4KI8yc3WyCWOrjHMXLpD3U282Jz0fuQ6yCjQQJhanJ8TRLPCYatqa/AfpgYXjYOsEm4SbkO4atjs/RMvpFuyX6cEHpJPS+QB8w+8kJNgT9h0ZKqbjwz8HvSbQPt6yCXSFrEo52jr0TBRgkHDMp28YDrLdw/F8kMbu79ja+sgu2jNtLVXS9t1Om0tqf79hiTxFKf85jgyawx1fYV2mdMmHrs046HN+9+wTWffca/vfsXzBz2oSoj8/Py8Hd11+EHz96Guu/fx1f/fdRHLTPbjCcqrHqLVVVGtEz0MiCGRGeS6RnYCyQaJyxn/r1ws9i+xWjek8alvPoTtx5pFnF0/ZWv178dVLrCL+mKPuIQqJn3QaceZsaOjeh0jop97pI6+9IeXWh52Uy60nUKTah0rqPeCRnT+RViuNA2xWuY8N2Q0PmEdGfP+N3UYVjd6faX5JE48cvBE+isni6jJ35jnqxNkmbbkMunx39Hf3r0ZxHEdJf2Gm/OEolxPEiat3A2hQIv4b2eEwV57FGuxzWtao7rUQbUWhXpbXtzuPRh+6DW686D9fd9RAWLF6F8087Gi88fDv2PeYiNDS19Hl8msuFlx69A/WNzbjgmnuxrbYBwyvL0dpmwviAKq3SumatGr5Wch6TqNxNplVPpAkzifQMjIVxM1THi8L1q+bH9Cupl/OYRMHMqCnq6ydxtHYRDCMWwTZ1tvqBgET0vsenXtjahHzHpJqEl5jkPNL/SBDSeshdpDC1mBlNx4gcTxrZaVXIOvSYibY8370DfPs27KQkBYplQsVIKuQX9lmPDVNuUhkhrgMh69TJeRQ9HlMv5xFB2NEgPCWcxwvOOBYvvP4BXv7vJ1i9bjN+f+fD6OruwSnHHhL28Scf+3MU5ufinCvvwg8Ll2NLTS3mzl+CZasMnuxAzpNwHkk8GlG5m1TBTMi7TlBIzYQ17XKgerv4q5jbjyRVvWtmzmMix2zaz9Tb5d8Z2xTe3zswyuesyXsGvt7tEKBijMlh6wTFmoEh64SrremxBSXq12ZMVBHnkAjT04cJEZ4mEW1lvqP+wwcVconq6nXW5zhGbBBuY7FMuDCo3ZXNoeLRzsrvVER/vFIhZJ3KzuNWf9g6+KQaLGHruJxHchGnTx6PB//9qv8+WZbx1XcLsdt0bT5rCIcesCfm/7RCCVsfdsCeaGhqxRvvfYGHnnwNPuEChOCUJDgpvBsHckkV3PQG3duFtKbt8HncoOPoTM+E02lCmDjcGtIz4dZERlpvNyTddtOcTng8vUqfx7SMTEg9xp1BlNvpHjtd3c7ir4K2Gwlaj7j4O9Iz4EpwH8kOJ9xaVWtaZ0tM2464Hu1CK6VlKN/HvIa0DLgn7q587Vr6LRwGHG+xfcnnUfr0OdMzwp5HvvKR8NDYP3cvpE3LII+b6U9dSOtIfH9E3T90DsXxvJ7SStBfmrNpu2F/C8p6tLBvPMfLVzEaigRub0G6uxswcD3KWrza8SoqV/7+KQ/V1bRD2aZUWq38jRKuus2GnCf9rcfp86rrGDFJTWnobENa3WbDzot41yNuhymfP7xo8jqQbvFa9OvwgW4DV9ylXU6kOx22rccbup5uWo/1Iin0eNmN/3hJdKt+iP2hw55jpV8P3VYVqjqivlWy5VwOOk5yYH/UNtP+6duWp73L/HUme/70er3Gisfiony4XE7UNQTnttU3NGP86OFhf2dUdQV+tvt0vPHu5zj9//6AMSMqcfcNFyPN5cR9j70U9nfyMjNRlB0yi7YfOqtHYwd9wG+pR3VBAeokGeT95eTkobBQ87VNxp1bhC2aI1Odmw0Jwa9hIzUMdqWjvLgE6Q7jwtatk2ajweFAxo4NqJJ7gBhfb4smRrKzclCW4D7y5BRiM+Vbej2oTndBSk98X3doYjY9MxtVcaynbcIs1JMAbq7F8K5GSAYe7wzIynyiorwC5IV53sYZ+4KSNbI3L0fJ3DexZeRkf6pElcMHh4Fr6dL2jysjA9VxPO/WsuEgmVfa3YZsA9fTrbl8zozMmNfTPGE66N0ju24jhpnwd+n0eZRLW05ZFajIMbOnE6XuduXvUnFgNZe9qrsZTgveF/JdTtTTF7lq5XdOzSqUF+qqwC2mPE8tIhqVQ7G0LvQ46dhp0RIbyMqkZMdef0i0NaMI1RlSSqxnq9uB7LyikHdxe45XqpCnXJd7lGO1I60I1ZrrZ+f+qSyic1mGz5uH6kJ7xXZuBhkHHejoBvIzC5GvfFaV0etpQ7qmtrw+ev/WWsql6PmzvqHB/mprySGhobEF19zxkOI0Ll6+FhXlJbj4zOMjise27m509saXzO4do2Zcuxt3YGtzM3ydqtxv98noaLbGz/YOUwuH5PZm1IRsU+8c7ejshsPANbkr1e16Vs5XXnss0HoyNTHS6ZNj/r1QfNlaeLCtCTXNiRfM0HpyNTHSKzniWo97zEzlVl78dZ/9nsx66A+vt1vtq9DU60ZryHOTw+Ueozq+PYu/wY6tG+H45k14DzhJcZi2NdQZshaxnoIy9RO/W3LGvH+o+MCdr+b/Nm5egyYD909BiXr+eJxpMa/HXaqGjXvWLUn4nIt2vLy9aqluW4YqiHpbG7Fj01rlw40s0jNa6rF9h7ljCcV62jqCE566V/1o6OuOdz21bW1we73IKFLPpY0dPdja7LFtPfWdgb4lS7skbGzumzdv5XpqdeuZ35b4+6LRx8tuxHq+qe/ChwUS5nVIWB+mxsHq9dS3t6I4l96JJSze0obaFnvX893aDnywUMbCDcHnTl2rhOpiVWhvaezG1ubuAX/+xCUeG5ta4fF4UVaiVddqlJYUoq4+vHCorWuCx+MJClGvXr8Fw8qKlTC4W+SU6fDKMrzxvmCtaENuqVcsV0evenC8zjT4rPrj2+OX6u2Sr8PaviJM7KHiCqPWRGFyKhah17pmYVz7LUvLofM502KyqcMipni0NSb+HCF9A8mdjfm58kuAUZOVL72Lv47/vOkHWRO0Xoez73NTbiP17OvtgXf1AvXn370PpGUBdZuT3h/Rii9ifm4qHKNWOl6P8sHKyHF8kkfrqRLr8aK85OHqBx3vxuWGHysFcQ7ReUHndnuLksKCxu1AmRYd2bHR8GMTCSFm/d+v/cmc1x0jdCGh117kVN+Pd/T6DK/di4cuGsGhsaA9tnCZqevx6NbTYf96xPFKFeh4Hb5chGbtX1dBjg8up0MZBbit2atMdLGTbrcP5zzad7/Utzn94rGhnf7m5AF//sSVsEBC76fla7DPHqrbQkiShH32mIH5P60M+zs/LFqG0SMrlccJxo6qwvbahrDCMWFEgnyL5vb424iYPA5QQJWUlWMUIYF5Hxk/ISQSIyaq4oDa5MRZBGBI43IDK4sTmt0861C1TdGGpYqjZDjRKpxFocyahYG2LyTOvnxVLdwxmISq42doE4dq1hk+x9nfJDzWAicqVqG8P5q+ZFaPxdCCmY6WvpXmVhXL6NdD0GtuT4Gsfl2fx3q3lDIFGItSoApVXzDDxTKpT1sXcO6jHlz7nP3CMRr1WqPwwTLXmog72/Wfz76JU48/DL866iCMHzMc9954CbKzMvHSfz9Wfv7AHVfi+t/+xv/4Z155D4X5ebjj2vMxdmQVDt53Fi4791d46pV3jX0lBWpoDs2qgJDcBk1PiZW9jgxMdonQS88/S9rINY1XQ7aJtKdJSKxFbNOT/Ci+uFv1kBAR4xi/fw+mEK2R+mStKfkK44Vi1GrrmPdPNrDbz9Wv575j/Hr0H9BIwMfyQYfYspoq7WAK4pwWXQ+odVVoZTfNkrYKfQ/HFKiyFpSkSp9HX2qJNb14/HGQVMUOZmgc4fsLZbz0rf1teqKhb9czJKutibc+/BolRQW45uLTUFZahKUr1+G0S25V+jgS1ZVl8OkuDDU76nHqJbfgtqvPw8f/+YfiOD7xwttKtbWh+J1HzX0SITUr+jxSi6DRU1Sh8f37ER9myohC6u9IrP0p7l81pFWPv0F4o/XikVw1EpD1W4G1i2EKkZzHKbPVc47SI4zsKxmL2KdUBXLy+xNgux2s7p/azao7avh6dK4aHTMtVSQiIyept5sMnAAUSmjDbb/zWGOT86hbz3qTztEknMeGFGnVQ2MAF6XARbXRowrYrb3qP4YxgjqtUbiYwT0YSKhg5smX31H+hePE827ocx+FtI/6zTUwDWq/ka1VFbVq4lELI1L+nGWu49JvgdaGGGYBGyQeadwatYkhEbhhibWzkk0IWwfCoDEIfsob3f1Q9evvyHU02cnS7yPK8zz0jICjZ9GEEP+HD+GERpvfTWJu98PUr+f8z5T9o3z4oDF/5DrSMYsqHqWA82jk+MhQQo9FpyYed2hhckrvMLj/ZlTcPYHbzdaPIQyHE7J/tnW9zc7jxh4JrR7V5etIgTGA63sknL5awrpuWov962EGBw3sPKYoWnK8Ei7u0arlrGoSXlQOTJylfj03eije8JzHcVruKV2U+nN9oq4nNcRjkJNFa4omyihkTMednKWlc5Ledlx5hr88Rw2LbluvCTNrCB6/lx5dPJIrS+P5mnaYkn+prEd8SKMPb/0JfpruQvuMcoJNDBvTORQkk0XYmtzpNx8yJy822npoig5FI8jt1It/GxGj98jto9nWdtLklTB6gQOdKZSv9lK9Pb0LmcFLna5R+GCZMDM4xKM2ilB/YRANjE2fbb3zvurtup/6nZhhSI6hHmpITaxNLCSZdNiaQqd5hcblPOovriRGoolHUdlOxUlmXpRDnced9wEm7KLe/79/Gl6EEg2JXD46ZhS2jlY0Q67snocHnFH6PbMgR00Rj+mxhay3rjZ3n4WeC0I8EiaJ6GhIJGU/eQGphAhZN3qoSbf97lqr1/41MIwVOY9dvTK6U+MzZNIMjo9YofmO+nCRqc6jFBiL99NX/T/aSOeRXpe4ICeQ72iImKXwLQkVEgMdyVeRSpTDF8uaqscDFaPVY/zjpzAVvXgk5+znp6vff/WG6mZZTSxuMYlbKmSiyt7FX5u7HvEhrb+/M5qwYna+I6H/wEFRiGju7BClNEVC1gwzVNhcrzqP25L3WFIG16AVj9pFQ0z7MIWRE9Vtd3cCqxfEMQvYgJxHuhiTgGiuS3hGcNJha+H+UWjUqOpZOm60HiFGSCiSCNIf22n7qLcrfohY2W4Y+rD19P2AzGw1f+47g7sFxApV7Kf3c8zEBxoSjkbO+Y4WJu7v74z+VojN4Vt6mSIeO3WuI+OnRDiPg8QBYZhUZ0MdcN6jHmxqGBzFMoNQPOomelgRtqYQJrHi+5iKJhLq0xcJrTE2Ni5P+CkcyYjH0VOBPTXx+Gn4SUEJQcctM0cVI5SOcPqNyrQWPHK1Kixp34n+ima7aoS+l+EuB6pfz/vQ3FBw1PX0c16TOyoq8Jd8Y/56/A5/FPFI68ktVNdO/SbNRP93KCqtmSBKXOoFjJ1HhrGO9xYOHuE4iMLWwT0e+/SgMwMSExN3j0vEBJzHdOPE46ZlCT+Fv+8kVctSHl2skEA58gL16wWfGNsGRn/caIILhcVJeAjhRn0ts3LUqvZNiQvnWPELfpqMQtXtXR3Asrmwjf4q5ElY07HcvsGasHp/H9KoG8DRF6lfL/rC9KKRoKIrfb4jEyZszbmGDMMMafEYLudR5GKZFLbeaZbaQ49CtltWxen0JSkeabvDRiftPAYVqMQjsqnamPo7UiXpJy8mvP2wa9IfN6rQFex1hLpGEZJd8q15jab1hLYPWvK1rXl0/X4o8u8fC1zH/nIeqZDmhCtUJ5n+Rgw+V/p3Hlk8pnKPR4ZhBi4DXzyG6/FIBE2/kMwLWcdxkfY7fcm6odQvz+FQ5/VS37pEoUIXEX6NNZQ+crLamohy6d56xHghpQ/LllQH7if3cfaRgfZEVomj0HSEBSYX6CTjPJLLRzmidFxNbF8UU9ia3GxyHOkDQGsj8Po/TM+/VNB/IBI9HpmUnC7DMMzAxTEoezzqHREz8h6pypgmysSZd2dYtfUobdsbEw9ZK+uh/8UbShfj7igEacakDr2TVVqpfi3C4vscp4Zkt61LuEgoKTFC+7tRNyPZDqLlqQrXcf0Sy4pFAlOBMkKE44XAhF3VDwOvP2Bd/qEYS0qw8xiW0jQt55GdR4Zhhqx4DNPjsY9jZHS7nuE7qRdIyiuLo+mwIRNdhPuXZMjaj3/kXQxrolD1TruqX89XZ5kbjhAj6VlAsSYeKdxJ7pXAKtdRWY8ndVxHvTjq8wFEAqbubV0hUaSWWJSjeuwl6vhGchrffFhtpm4VerHPBTNRnUfOeWQYZuiKx3D5jlpz3rCuiFGzrImatXH9miHOI+WPDRthXM+8eJzHXQ5SxQE5cGYVYwjnsaxaFdkkTig8T82uCRIkFhasSO1Nahi4pQFYNR+2E8l5pHOSPkiR+776R+vWE5RbLAHHXgpM2kM9r17/e0wtrIyEC2Zib9XDOY8MwwzdVj354cWjvwcdCTWjw9aVQjzG13bEkGpragxOrieJNyOclVjdUAoXzzxA/Xr+JzC9byA1AScatqtzmRd+DpSPAOq2qK17LEKiHpPP3KGGgS2cJhNxPR63un9Cz2mqQhcN460s6PHonMdxO6v5sLT91x4A1i22bh3+9XCfx1irrTnnkWGYoSsew/V47CPWDHQeqfimcoz6NeXeWT1L2oD+jmHFWn9ha2pLRLmeFD42000SYdBho9TbBs3hpH333r9hC3EeZ1OJVG0tejsmOKoyUQJjQDOAadqozh8/s0c4EkHOI4etQ3FJMgp5wgzDMBjq4rGwtE+PR4HD0wuv0b0eS6rUCm8KD8ZZtGHIbGsj8x0Jb4xTb0ShzMLPzHXgxD6ifUxQOyAmulNMlejk1FLlPM1YtxIh9ikfVkyRsTLnMlKOKolIfQEdo1DsVG+9MtDM4pFhGAx151HfpsdM57FqnHq7fX3cfQaTmuhCUEsiCt0SRjXI9hfMpAeECAkC/YW3tFptkk35hhQ+NhN9lTzB4jGYcHmzwnWkwhQLQ/pBx4smDlH7KEorMKMKP0Ykeh+gc5pGSDJ9cEjAaw1AugT41H4LDMMwQ0w8kijMzle/poIGMye6JJnvaIjzOE7La9u+0biZznoni4TjhX9WHdWnbg08Zqfd1FsKRZocCpREDp3AqpY8AwV3mA8gQjwaOekn3jAxCUe7XUc6f0g8P3oN0NNp6zpSle1uCSet0uxHhmGYIVdtTVW/B56k6/HY92IRqLZON77SOoE8uKRb9VAxArFqHgxDvyZyF9Mz1JxO8Tr14tGKamO980hOJ03wYSLPR6dCJnL9iLWLrF+PCFsTPh+w9FvYTlsj0Ntt9yoYhmEGLQPTeSSH7Lj/U/stEl+9EfZhlPPoT+Y3AhJYImyciPPozy9MQMxSDuCYaerXK40Tj/6CGXptomcmQX366DXmFatikvLp1vxobbUsCccUqHBOKUI/ENG0IRpXSVXh5EhbjV7sb1iiroNhGIYZ1Aw855FE1G9uVYVjdwfwn/uA+R+FfajhYWuqACbHky6Q5G7EiSTGs9FzULudeKDQJAm8hm3G9ljUh9LLRwbun7ynWlk+YRf1+y1rrMmn0ztZnO/Yf1N3f5U1uY4WzPqOJh5tDlkzDMMw1jDwnEfqc1hQos50fv7uqGFNw8PWolgmAddRXU/I1Jt4Qms7zTLcdVTQF/HoxSO5uzQG0cqQdWiTZ8537Eto6oPo77jG+pA1IYleit2dwCprG4IzDMMw9jDwxKMoWKH5vf3kwzmSrbYmh/DI89VJFV+/Gdj2tvgmy/QJW4uLf6zikVym8TOMz3fUF2CQWCRRTiybo4atdz1YFeuEVZNC9E4WO4998Z/TaWp7nuIKNTeUQsY2IDXXAm89AjTXWducnGEYhrGNASgeY2/Q7XexEq1urh4fmBdMYVynMznnUYgjch3jWdOYqWq4vrXB+DnBQtCKAhkSATRBhsSjKNCh9itWFa6w8xh7xf7PT1O/XvG9vQUiS+fYt22GYRjGcgageBTuX/8iKumcx7Lhga+pCbKAejwmI47iFY8TTQpZi/UQImRduxnYslptfSScSCvDkcJ5pAIdyu9kIoj9cWqlNeWIfvaK3atiGIZhhhCOAdcQnBplU5iudlMc4wCTFI8/fKCGrWm7m1ep+V2JEm+jcAqdT9jVNPHoF9gkRIi6zWrhBYWuBVaFrGk9NKOcBC25uxwGjSyuxfH69u2EircYhmEYZmg4j8J1JHdMVC5Hwd+qJ1HnkSarCJeT+tfN+xDoDWlinWiOYaxroqryrFyAChO2rILh6It4CDGZY8k3wJ6/BEjMGR0qj4LU1QY8cjWPlouEPm+WUgy+f8/O1TAMwzBDkIEpHmNs0B2ots5IznmknD/CgKkuQX0VY0G0yqHpIXGOQ0xIPApHl9oBPf0HrT2PxS1guFdgbMfrkxf7Hj+GYRiGMRnXwCyWWW/+OECqPibHz2dw7l28axLicbVJo+f04oNc1abawPfbN5izTSZhJBL1W9eo4t7oynuGYRiGGVTikRpWU2uShJzH9MRdx6btwaHCZImnArykCigapgq89YuNW0O49ejzHZmURWk0/8ztdi+DYRiGGcIMnIKZ4kp1DBu5YzH2/wv0eUxAPJaGhKyNIp751sJ13LgsePKKgQT1noyhCIlhGIZhmKHNwBGPog/hjg1qGxezcx79+Y4GjgKMN2ztD1mbOFNaH7amQiSGYRiGYZhBIR7jzHdMukl4WbUpzmNA0PbjPFJLImpSTqwxUzz29q20ZhiGYRiGGfjiMb5K66CwddziUTIxbB2joB03A5AcatEKzfE2C48nJOeRYRiGYRhmoBfMUKNsMQElDvGY8ISZwlIgPUMVekaP5YtV0IrG4GaGrP1Nud1qvqOdI+4YhmEYhhkQpKZ4LCoPbhkzYie1wIT6LOrvjzlsnaa6eDHmSvpdx/ptsf9OogUzw0apDcD17iIJyzHTzA9Zi6bcj3JTboZhGIZhBnLY+qK/BM+VnnmAerv8+7iexu88xus+im3XGxyyJvSClirIz7oNOPeu4Nd7xLlAeqY6QcSKXoskXNl1ZBiGYRhmwIpHYtah6m1WHjBxd/XrhZ8l3oYmnrzH0MkyBhLUuHzyHmpInpqRn3wtUFgO/OwYYMpsdfziO48bvn2GYRiGYZjBKR6n7q2Kqp33AZwuNddxx8a4nkKi/4lZ1HE5j+ZUWivoG5fvtJv6NYWMaaLNGTcB+52g3vfBM8CmFcZvn2EYhmEYxmrxeNZJh+O7d5/Auu9ew/+e/QtmTpsQ0+8dc9i+qFn4Nv79txujP5Da8ZC4mnkgsMuB6n0/xuc6+om31yM5gTTZxSzx6NbWU1KpTsyh8Yc0MaRxuyogie/fBxZ9bvy2GYZhGIZhrBaPRx+6D2696jzc99iLOOyUK7Bs1Xq88PDtKCkqiPp7w6vKcfPvzsHc+Uv638j8j9Tbnx0NFFeoztyyuUgId5zOI4k6cjppm60NMBwKR+tbD21Zpc4pfulP6sziRV8An75k/HYZhmEYhmHsEI8XnHEsXnj9A7z830+wet1m/P7Oh9HV3YNTjj0k8kYcDjx091X46yMvYOPWGFrfkFDsaAm4hUu/TXw8X7yNwkdOUm9r1ia2vVjXI1g1X72lljnkQL77L+MrvBmGYRiGYexo1ZPmcmH65PF48N+v+u+TZRlffbcQu02fGPH3fnfhyahvbMGLb36EPXad2u92nD4v5IWfw0fFI7TIn76Ew+mMZ6lI0x5PjcJJirkyMmN6DveoKZBpDZtXwBnnNmNZj8vngVd//9qFkAzcTrzrEbd2w+uJDq8nOrye6PB6osPriQ6vZ2itp9erVykGiMfiony4XE7UNQRPPKlvaMb40bpWMzr2mDkFJx97CA496fKYt5OXmYn8dfOwdcZ+yKjfgmG9rUChlg8YJ2myF+RZFhcUIaef55AhYdOoyYp4LG/cgswEtxmNwjQXhPeaXr8V1ZIn4ddmBOV5eUgleD3R4fVEh9cTHV5PdHg90eH1DI31rG9osLdJeE52Fv5+1+9wze0PorG5Nebfa+vuRue2LXA+/DtQhuDWBLZNipt2nLtbbX7d0OtBc3Nz1N/xlY+ELzNHyXesX70YkoHhY7GeltYW/33eFT9gaz9rMguxntq2Nrhj+JTB6+H18Hp4PbweXg+vh9cTt3hsbGqFx+NFWUlR0P2lJYWoq+87f3n0iAqMrB6Gpx+42X+fw6E00MGmeW9i32MvwsYt2/v8nleW4TXohJC1XEmv09X/cw7XQu+bV8KtbzBuIB7ROojWtPIHw15notAfXiwWtVXweqLD64kOryc6vJ7o8Hqiw+uJjnsIrScu8ej2ePDT8jXYZ4/peP8ztfpZkiTss8cMPPXSO30ev2b9Fhx4wqVB9/3+/85QHMlb/vRP1Gyvh9lI7l4lDB1Tq57RU9TbjcvMW09Lndp7sqEGqN1s2nYYhmEYhmHMIO6w9T+ffRP333ElFi1bgx+XrML5px2D7KxMvPTfj5WfP3DHldhe24B7/vEMenrdWLl2U9Dvt7R1KLeh95tGpGrrjGzg5GuAhm3A//6pzr4eoTmPG5ebthyJ5nPTLGmTnE2GYRiGYZiUEo9vffi10tPxmotPQ1lpEZauXIfTLrkV9Y1q7l51ZRl8suL1pQaR+jxS8/Gqceo/asuzfT2QkQWQuNthsrClNkQMwzAMwzADkIQKZp58+R3lXzhOPO+GqL975S33w1LERBd92JqmyOz288D3B50MLP9e/VoZCZhC4pdhGIZhGCaFSN3Z1gYh+cPWaYE7J84C8ktUB3D9UlVYTt/X9HxHhmEYhmGYgc6gF49hZ1vvfph6u+ATNd+RQtUCE/MdGYZhGIZhBjqDXzy6Qwpmqser/6hgZcGnQHsT8MHT6s9aGtQ50wzDMAzDMIz1TcJTAn/OoyYeZ2mu49I5QKfWuHz5d0B3B9DaaNMiGYZhGIZhBgauoRO2TgeGjQImzVK/n/dB8OPWL7F+bQzDMAzDMAOMQR+2lkSrHiqQOfEKtdJ65Txu0M0wDMMwDJMAQydsXTRMva2vAd553NYlMQzDMAzDDFQcQyZsTVBV9at/A3q67FwRwzAMwzDMgGXQi0dlHCDh8wJvPAg07bB7SQzDMAzDMAOWQR+2lkgsvv8k0FTLDcAZhmEYhmGSZNCLR4UfP7N7BQzDMAzDMIOCQR+2ZhiGYRiGYYyDxSPDMAzDMAwTMyweGYZhGIZhmJhh8cgwDMMwDMPEDItHhmEYhmEYJmZYPDIMwzAMwzAxw+KRYRiGYRiGiRkWjwzDMAzDMEzMsHhkGIZhGIZhYkZC+W5y7A9nGIZhGIZhhjLsPDIMwzAMwzAxw+KRYRiGYRiGiRkWjwzDMAzDMEzMsHhkGIZhGIZhYobFI8MwDMMwDBMzrtgfyuy561Rccubx2HnyOFSUl+CcK+/C+5/N9f+8tLgQN15xFvbfayYK8nIxd8ES3PTHx7B+0zb/Y0YNr8AtvzsHe8ycgvT0NHz27QLcdO9jqG9s9j/mqftvwtSJY1FSXICW1nZ89d0i3PXAU9hR14hUxqr98927T2BE1bCgbd/9wNN48MlXMdT3z+xZ0/DaE/eE3f4vT/sdFi1djaF+/uw8aRxuvOJMzJg6AV6vD+9+8i1u+8u/0NnVjVTl/845EYcfvDfGj65Gd08v5i1agbvufwprN271PyYjPQ23XnUujj5sX+Xrz7/9Edff/UjQa6+uKMM9N16Mn82ajo6uLvzn7U9x99+fVvYDUV5apDzH9CnjMWZEJf714tu49c9PINWxav/QeUXnzrjRw5GVmYGt2+rw7Gvv4/Hn/otUxqr9E+n9Z8bBZ6CuIfA8Q3X//O32K3DS0Qf32f7KtZtw4AmXYiDBzmMcZGdlYumq9bjhnkfD/vzff7sRo6qH4ewr78KhJ1+OLdvq8PKjdypvMgTdvvjI7ZBlGb+64EYcc9a1SE9z4em/3wxJkvzP8828xbjw2j9i32MvwvlX34PRIyrw+F+uQ6pj1f4h/vTQc8obkvhHF7lUx4r9M2/hiqD9Qv+ef/0DbNyyPaWFo1X7Z1hZMV567A5FcB55+tU47dLbMHHcSNx/+xVIZWbvNg1PvfwOjvzNNTj5opvhcjmV1ypeO3Hb1efhkP32wIXX/BHHn3u98lr/dd/1/p87HA48849blH1y9FnX4PKb78evjzoY11xymv8xJLgbmlrwwOMvY9mq9RgoWLV/6APGky+9o/z+/sdfgvsffxm/v/R0nHbCYUhlrNo/gn2OvjDoPai+sQWpjFX755Y//TNov+x26FlobG7F/z76GgMN7vOYIDUL3w5yRsaOrMLXbz2GA064FKvWblLuowvWok+ewb3/eBYvvPEh9p+9C5578FZM3u8UtHd0KY/Jy83G8i9fxCkX36I4jOE4dP89lAvn6D2Oh8fjxVDfP+Q8Pv78W3ji+bcwULHq/KE3wQUfPoV/v/g/5UI31PcPXeSvveQ0zPz5mYrIJCaNH4VPX30Qex91ATZsDriYqUxxUT6WfPY8jjvnOny3YKnyOhd/9hwuvf4veOfjb5XHjB89HF+++QiOPONqLFi8Egf+bDc88/ebscshZ/ndkjNO/AVuvPws7Hzg6XB7PEHbePWJu7F05boB4TzasX8ET/z1enR29eCym+7DUN8/wnmctO/JaG3rwEDFqvPnFwfupZw/ex5xnuJiDyTYeTQI+sRO9PT0+u+ji1Nvrxu77zJFfUyaC3S9ovsE9HifT8Ye2mNCKczPxfGHH6DY6ANFOFqxf/7v7BOx5PPn8eFL9+PiM4+D0zmwT2Wzzp9D998TRQV5ePm/H2MgY9T+yUhLg9vt8QtHgsJURKR9mIrk5+Yot80tbcrt9MnjkZ6WFvQBYs2GLdhSU4vdZkxSvp81fRJWrNkYFGaj0Ft+Xo7ivg4mrNo/0yaOxawZkzF3/hIMJMzePx+9/AB+/OhpvPTo7dh95mQMNKw6f0459hDlOQeacCQG9hU3hRAn0vWXnYmCvBykuVy49KwTUFVRhmGlRcpj5i9eqYQ9KG+L7HD6R/lZ5A6VlxYHPd+Nl5+JNXP+g2Vfvqg8x9lX3ImBjJH7518vvI2Lr/sTfnX+jXj21ffx23N/jZuuOBsDGaPPH8Epxx2Cz+f8iG21DRjIGLV/vv7hJ5SVFCkfOOg56LluuOxMf77fQIAc1z9ccz6+/3GZkisl1t7T6+7j9tQ1NqO8pFD5uqy0sE/embjQlQ2Q154q+2feB09i/fev470X7lPCneR8DxTM3D+1dU249o6HcN5V9ygpVzXb6/Hq43crecYDBav+voaVFStu5UA6d/SweDQIcgXPvepujBtVheVfvYS1c1/F3rvvjE++nqc4H0RjU6uSy0h5E6u/fQUrv34Z+Xm5+GnZGvh8akKt4JGn38ChJ12u5F/Qzx6480oMZIzcP/987r+YM28Jlq/eoIjH2//6L5xz8pGK8zRQMfr8ISrLS3DA7F3w4hsfYaBj1P6hkPcVt9yPC884TnmOhZ88i801O1Bb3wRZe55U5+7rL8Kk8SNx8e//ZPdShuz+Oe7s6/DLU6/E7+96GOeddjSO/cV+GCiYuX+owOS5197H4uVrlWjZ7277u3J7/unHYKBg1d/Xr446SBGj738aKAocSAzcq20KQn8wh5x0uZIfkZbmUi5m/3v2L8rFS/DFnB+V3Kriwnx4vF7l5Fn48TPYtHV70HNREi39W7epBqvXbcb8D5/CbtMnYv5PKzFQMXL/6FmwZJXyfFSBra+OG+r756Rjfo6mljZ8+MV3GAwYtX/eeO8L5R9Vb5NTSSHsC04/BhujnGOpwl3XXYhD9tsdx51zfZCbTOKXKkApRKZ3R8qKC1GruSF19c3YZdpOQc9H+0D9WRMGA1btH/rAQVCYkp7jqotOwZvvf4lUx47zZ+HSVdh95sBICbFy/5x87CF49Z3PIubSpjrsPJpAW3uncmEbM7ISM6aMxwef9714kzCkk/Bnu09HaXEBPvz8+4jPR1Vc+ryvgY7R+2fqxDHwer1BuSYDGaP2D4nHV9/+bEDnypq5f+h8IfF4zGH7KiGpL+cuRKpf2H5x0GylklyIF8FPy9eg1+3GPnvM8N83blQ1hleVY/6iFcr3835aoRQHlRQV+B+z3+yZyn5atU4Nzw1k7No/9P48EN6b7do/9P5cW5/abeas3j+zZ01TigBfHKAha4KdxzhbidAFSzCiepjyh9Hc0o6t2+tw5CE/U9pcUPLr5Amjcfu15+P9z75T3BDBScccjNXrtiiP2236JOUxFIYVjhl9cpk5dQK+X7gMza3tGD28EtdeehrWb6rxn6RDef+Q+7rLzhPx7Q8/KRW1lKz8h6vPw2vvfo6WFK/us2L/CPbZY7rS83Ag5dNYtX/OPukIJZTW0dmlvLnffMU5Si+2VK4OvfuGi3HcL/fD2VfcpZz3ZVqeFQlpKvihW0pPuO2qc5Uk/7aOTuViOG/RcqUSlKD9tGrdZvzjrt/hzvufVHI/qc3MU6+8g153wP2gfU7kZGUqF0L6nn5OEZChvn/OOulw5fyjHFxir12n4aLfHJfyrcKs2j8Uwt+8dYeSK0hO3anHH6p8gKNuB6mMlX9fxCnHHor5P63w51QORLhVTxxEaoD68luf4Mpb7se5pxylJOKXlhQqicP/+d+nuP+fLwfZ0pSc/+ujD0ZhQS4219Ti2f+8p1zcBPTJhS54U3Yao1xMyS7/7Jv5eOCJl7G9NrU/vVmxfyjx+u4bLsL4McOV6jd6oyLr/5/PvtnnD3Qo7h/BQ/dcjeGVZTjmrN9joGDV/nngjitx8L6zkJOdhTXrt+DRZ97Aa+98hlRvXRQOyt985a1PgpoYH/OL/bQmxguUJsb6JP7qyjLce+Ml2Hu3nRXXlZoY3/X3p/xNjCNti5yYPQ8/D0N9/1Bu9ekn/gIjq4cpjj71T6U+qpR7ra/gH6r755Kzjsdpxx+mNPnv6u5R8tL/9thL+HbeYqQyVv595eVmY+FHz+DmP/8TL7w+cD7ch8LikWEYhmEYhokZznlkGIZhGIZhYobFI8MwDMMwDBMzLB4ZhmEYhmGYmGHxyDAMwzAMw8QMi0eGYRiGYRgmZlg8MgzDMAzDMDHD4pFhGIZhGIaJGRaPDMMMWmjmcKQGwHaRimtiGIaJBxaPDMMwIZz568OVSTWJkpWZoYhEmprDMAwz2GDxyDAMY4p4PBV7z9q5z8/uf/xljNnj+CRXyDAMYx8uG7fNMAwz5KA5t/pZtwzDMAMNnm3NMMygYI+ZU3DbNedh0vhR2F7bgIefeh3DyooUB7Bq5lHKY0465mCccMSBymPycnOwcfM2/Pul/+GZ/7znf57v3n0CI6qGBT33t/MW48TzblC+zs/LUULSRxy8N0qKC1GzvQ4vvP4hHn76dciyjOFV5fj+3X/1Wd9fH30Bf330ReV39WsiKAfyyZf+hznzl+Dqi07FiOphWLpyPa6940GsWLMRp5/wC1x85nGoHFaKBYtX4opb7seWmtqg599l2k64+uJTsdv0SUhzubBw6Wrc++Az+GHhcsP3NcMwQxt2HhmGGfCQGHzxkdvR0NSC+x59EU6nQxFSdQ3NQY/7za8Ox6q1m/DhF9/D6/HikP33wL03XgKHQ8JTL7+rPObWPz+BO39/ATo6u/HAE68o99U3NvvD0a89cQ8qy0vw7GvvY+u2OsyaOQnXX/YblJcVKb/b0NiC39/5EP5406V495Nv8e4nc5TfXb56Q9TXsMcuU3Ho/nviqZffUb7/v3NPxDN/vwUPP/0azvz1EXj6lXdRkJ+LS846Affddhl+fcFN/t/92e7T8dxDt2Hx8jW477EX4ZNlnHT0z/HKP+/Ccef8HguXrDZ4jzMMM5Rh8cgwzIDnmktOozgKjjvnOmzdXqfc984n3+LT/zwY9LgTzr0e3T29/u+ffPkdPP/Qbbjg9GP94vH9z+bi2ktPR2NzK15/9/Og37/g9GMwekQFDj35cqzftE2577nX3seO2kZcfObxeOyZN1Gzox7vfPytIh5JMIY+RyTGja7Gfsdd7HcUm9va8eeb/w+Xn3cS9jnmInR0din3kzC+7NxfKw6neOy9N12Cb3/4Caddepv/+Z579X189tpD+P2lZ+CUi29JaL8yDMOEgwtmGIYZ0DgcDhwwe1d88Nlcv3Ak1qzfgs/nLAh6rF445uVmo7gwXwkVjx5RqXzfH0cesg++W7AMLa0dyu+Kf199txAulxN77jY14dfx9feLgkLRPy5eqdySeymEo3r/KuV2VHWFcjtt4liMG1WNN977ImhN2VmZynPuuetUSJKU8LoYhmFCYeeRYZgBTUlRPrKyMvxOoJ61G7bi5/vu7v9+95mTlZzC3WZMUsSVnvzcHLS1d0bd1tiRVZg6cQyWfP582J+XFhcm/DooBK6nVVtLzfb6kPs7lFsKYRNjRlUpt3+/83cRnzs/NxstbervMQzDJAuLR4ZhhgSjhlfg5cfuxNoNW3DbX/6Fmh11cLs9OGifWbjwjGMhOfp35+gxX8z5EQ8/9VrYn6/bWJPw+ny+8BXY3gj3CzPRoX1x+33/xtKV68I+tqOrO+F1MQzDhMLikWGYAU1DUyu6unowZmRl2DxCARXHZGak46zL7wwKb++9+/Q+v0dV0+HYuGU7crIz8dV3i6KuKdLvm8GGLduV27aOzn7XxTAMYwSc88gwzICGHDvKbTzswL1QXVHmv3/8mOFKLqT/caK3os5gpDzHk8I0A+/s6kZBXk6f+9/+8CvMmjEZ+8/epc/PqIUPFbMQXd09/vvM5qdla7B+Uw0u+s1xfULxRHFRvulrYBhmaMHOI8MwA56/PPICDth7V7zx73uVljZOlxPnnHwkVq7dpOQoEhRu7ul14+kHblYqpHOysnDq8Ycq7X0qykuCnm/x8rX4za9+icvP+zU2bN6G+sYWfPPDT3jk6TeUdjrUQueVtz9RhBsJtkkTRuPIn++NPQ8/T6nSpsIc2vbRh+6rhLKbW9qUfo10n9GQy3n17f/Acw/ehs9fewgvv/UxttU2KO2E9p41He0dnTjz8jsM3y7DMEMXFo8Mwwx4qCXOqZfcituuOhdXX3Iatu2oVwQlNQkX4nHtxq244Op7cO2lZ+DmK89RekA+8593FfH4tz9cEfR89z32Eqory5WeiuROUpNwEo/kKB5/7vW47LxfKZXXJx55ENrbO7Fu01Zle6KYhbj6D3/Hnb+/ELddfR4y0tOUJuFmiEdizrwlOPrMa3DF+Sfj7JOORHZ2JuoampTK7Gdffd+UbTIMM3ThCTMMwzAMwzBMzHDOI8MwDMMwDBMzLB4ZhmEYhmGYmGHxyDAMwzAMw8QMi0eGYRiGYRgmZlg8MgzDMAzDMDHD4pFhGIZhGIaJGRaPDMMwDMMwTMyweGQYhmEYhmFihsUjwzAMwzAMEzMsHhmGYRiGYZiYYfHIMAzDMAzDxAyLR4ZhGIZhGCZmWDwyDMMwDMMwiJX/B4RNlEnzJ7ASAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 700x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Train-val-test dates\n",
    "# ==============================================================================\n",
    "end_train = '2001-01-01 23:59:00'\n",
    "end_val = '2006-01-01 23:59:00'\n",
    "\n",
    "print(\n",
    "    f\"Train dates      : {data.index.min()} --- {data.loc[:end_train].index.max()}\"\n",
    "    f\"  (n={len(data.loc[:end_train])})\"\n",
    ")\n",
    "print(\n",
    "    f\"Validation dates : {data.loc[end_train:].index.min()} --- {data.loc[:end_val].index.max()}\"\n",
    "    f\"  (n={len(data.loc[end_train:end_val])})\"\n",
    ")\n",
    "print(\n",
    "    f\"Test dates       : {data.loc[end_val:].index.min()} --- {data.index.max()}\"\n",
    "    f\" (n={len(data.loc[end_val:])})\"\n",
    ")\n",
    "print(\"\")\n",
    "\n",
    "# Plot\n",
    "# ==============================================================================\n",
    "set_dark_theme()\n",
    "fig, ax = plt.subplots(figsize=(7, 3))\n",
    "data.loc[:end_train, 'y'].plot(ax=ax, label='train')\n",
    "data.loc[end_train:end_val, 'y'].plot(ax=ax, label='validation')\n",
    "data.loc[end_val:, 'y'].plot(ax=ax, label='test')\n",
    "ax.legend()\n",
    "plt.show();"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "8bfc04a8",
   "metadata": {},
   "source": [
    "## Grid search\n",
    "\n",
    "Grid search is a popular hyperparameter tuning technique that evaluate an exaustive list of combinations of hyperparameters and lags to find the optimal configuration for a forecasting model. To perform a grid search with the **skforecast** library, two grids are needed: one with different lags (`lags_grid`) and another with the hyperparameters (`param_grid`).\n",
    "\n",
    "The grid search process involves the following steps:\n",
    "\n",
    "1. `grid_search_forecaster` replaces the `lags` argument with the first option appearing in `lags_grid`.\n",
    "\n",
    "2. The function validates all combinations of hyperparameters presented in `param_grid` using [backtesting](../user_guides/backtesting.html) or [one-step-ahead validation](../user_guides/hyperparameter-tuning-and-lags-selection.html#one-step-ahead-validation) validation.\n",
    "\n",
    "3. The function repeats these two steps until it has evaluated all possible combinations of lags and hyperparameters.\n",
    "\n",
    "4. If `return_best = True`, the original forecaster is trained with the best lags and hyperparameters configuration found during the grid search process."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "748732ed",
   "metadata": {},
   "source": [
    "<div class=\"admonition note\" name=\"html-admonition\" style=\"background: rgba(0,191,191,.1); padding-top: 0px; padding-bottom: 6px; border-radius: 8px; border-left: 8px solid #00bfa5; border-color: #00bfa5; padding-left: 10px; padding-right: 10px;\">\n",
    "\n",
    "<p class=\"title\">\n",
    "    <i style=\"font-size: 18px; color:#00bfa5;\"></i>\n",
    "    <b style=\"color: #00bfa5;\">&#128161 Tip</b>\n",
    "</p>\n",
    "\n",
    "When using <b>backtesting</b> as the validation strategy, the computational cost of the tuning largely depends on the strategy used to evaluate each hyperparameter combination. In general, the more re-trainings required, the longer the tuning process will take.\n",
    "\n",
    "To speed up the prototyping phase, a two-step approach is recommended. First, run the search with <code>refit=False</code> to explore a broad range of values quickly. Then, refine the search within the most promising region using a tailored backtesting strategy aligned with the specific needs of the use case.\n",
    "\n",
    "For more guidance, refer to the following resource: <a href=\"../user_guides/backtesting.html#which-strategy-should-i-use\">Which backtesting strategy should I use?</a>.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "10187b4f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e115f0b17819498185a7b88d66e1e9a6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "lags grid:   0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b61c73e2169d454793964b1d4499e190",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "params grid:   0%|          | 0/6 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "`Forecaster` refitted using the best-found lags and parameters, and the whole data set: \n",
      "  Lags: [1 2 3] \n",
      "  Parameters: {'max_depth': 5, 'n_estimators': 100}\n",
      "  Backtesting metric: 0.04387531272712768\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lags</th>\n",
       "      <th>lags_label</th>\n",
       "      <th>params</th>\n",
       "      <th>mean_squared_error</th>\n",
       "      <th>max_depth</th>\n",
       "      <th>n_estimators</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>lags_1</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 100}</td>\n",
       "      <td>0.043875</td>\n",
       "      <td>5</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>lags_1</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 100}</td>\n",
       "      <td>0.043875</td>\n",
       "      <td>10</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>lags_1</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 100}</td>\n",
       "      <td>0.043875</td>\n",
       "      <td>15</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>lags_3</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 100}</td>\n",
       "      <td>0.044074</td>\n",
       "      <td>10</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>lags_3</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 100}</td>\n",
       "      <td>0.044074</td>\n",
       "      <td>5</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>lags_3</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 100}</td>\n",
       "      <td>0.044074</td>\n",
       "      <td>15</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>lags_1</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 50}</td>\n",
       "      <td>0.045423</td>\n",
       "      <td>10</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>lags_1</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 50}</td>\n",
       "      <td>0.045423</td>\n",
       "      <td>15</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>lags_1</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 50}</td>\n",
       "      <td>0.045423</td>\n",
       "      <td>5</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>lags_3</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 50}</td>\n",
       "      <td>0.046221</td>\n",
       "      <td>10</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>lags_3</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 50}</td>\n",
       "      <td>0.046221</td>\n",
       "      <td>15</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>lags_3</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 50}</td>\n",
       "      <td>0.046221</td>\n",
       "      <td>5</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td>\n",
       "      <td>lags_2</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 100}</td>\n",
       "      <td>0.047896</td>\n",
       "      <td>10</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td>\n",
       "      <td>lags_2</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 100}</td>\n",
       "      <td>0.047896</td>\n",
       "      <td>5</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td>\n",
       "      <td>lags_2</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 100}</td>\n",
       "      <td>0.047896</td>\n",
       "      <td>15</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td>\n",
       "      <td>lags_2</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 50}</td>\n",
       "      <td>0.051399</td>\n",
       "      <td>5</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td>\n",
       "      <td>lags_2</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 50}</td>\n",
       "      <td>0.051399</td>\n",
       "      <td>10</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td>\n",
       "      <td>lags_2</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 50}</td>\n",
       "      <td>0.051399</td>\n",
       "      <td>15</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                               lags lags_label  \\\n",
       "0                         [1, 2, 3]     lags_1   \n",
       "1                         [1, 2, 3]     lags_1   \n",
       "2                         [1, 2, 3]     lags_1   \n",
       "3                     [1, 2, 3, 20]     lags_3   \n",
       "4                     [1, 2, 3, 20]     lags_3   \n",
       "5                     [1, 2, 3, 20]     lags_3   \n",
       "6                         [1, 2, 3]     lags_1   \n",
       "7                         [1, 2, 3]     lags_1   \n",
       "8                         [1, 2, 3]     lags_1   \n",
       "9                     [1, 2, 3, 20]     lags_3   \n",
       "10                    [1, 2, 3, 20]     lags_3   \n",
       "11                    [1, 2, 3, 20]     lags_3   \n",
       "12  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]     lags_2   \n",
       "13  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]     lags_2   \n",
       "14  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]     lags_2   \n",
       "15  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]     lags_2   \n",
       "16  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]     lags_2   \n",
       "17  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]     lags_2   \n",
       "\n",
       "                                    params  mean_squared_error  max_depth  \\\n",
       "0    {'max_depth': 5, 'n_estimators': 100}            0.043875          5   \n",
       "1   {'max_depth': 10, 'n_estimators': 100}            0.043875         10   \n",
       "2   {'max_depth': 15, 'n_estimators': 100}            0.043875         15   \n",
       "3   {'max_depth': 10, 'n_estimators': 100}            0.044074         10   \n",
       "4    {'max_depth': 5, 'n_estimators': 100}            0.044074          5   \n",
       "5   {'max_depth': 15, 'n_estimators': 100}            0.044074         15   \n",
       "6    {'max_depth': 10, 'n_estimators': 50}            0.045423         10   \n",
       "7    {'max_depth': 15, 'n_estimators': 50}            0.045423         15   \n",
       "8     {'max_depth': 5, 'n_estimators': 50}            0.045423          5   \n",
       "9    {'max_depth': 10, 'n_estimators': 50}            0.046221         10   \n",
       "10   {'max_depth': 15, 'n_estimators': 50}            0.046221         15   \n",
       "11    {'max_depth': 5, 'n_estimators': 50}            0.046221          5   \n",
       "12  {'max_depth': 10, 'n_estimators': 100}            0.047896         10   \n",
       "13   {'max_depth': 5, 'n_estimators': 100}            0.047896          5   \n",
       "14  {'max_depth': 15, 'n_estimators': 100}            0.047896         15   \n",
       "15    {'max_depth': 5, 'n_estimators': 50}            0.051399          5   \n",
       "16   {'max_depth': 10, 'n_estimators': 50}            0.051399         10   \n",
       "17   {'max_depth': 15, 'n_estimators': 50}            0.051399         15   \n",
       "\n",
       "    n_estimators  \n",
       "0            100  \n",
       "1            100  \n",
       "2            100  \n",
       "3            100  \n",
       "4            100  \n",
       "5            100  \n",
       "6             50  \n",
       "7             50  \n",
       "8             50  \n",
       "9             50  \n",
       "10            50  \n",
       "11            50  \n",
       "12           100  \n",
       "13           100  \n",
       "14           100  \n",
       "15            50  \n",
       "16            50  \n",
       "17            50  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Grid search hyperparameters and lags\n",
    "# ==============================================================================\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator = LGBMRegressor(random_state=123, verbose=-1),\n",
    "                 lags      = 10  # Placeholder, the value will be overwritten\n",
    "             )\n",
    "\n",
    "# Lags used as predictors\n",
    "lags_grid = {\n",
    "    'lags_1': 3,\n",
    "    'lags_2': 10,\n",
    "    'lags_3': [1, 2, 3, 20]\n",
    "}\n",
    "\n",
    "# Estimator hyperparameters\n",
    "param_grid = {\n",
    "    'n_estimators': [50, 100],\n",
    "    'max_depth': [5, 10, 15]\n",
    "}\n",
    "\n",
    "# Folds\n",
    "cv = TimeSeriesFold(\n",
    "         steps              = 12,\n",
    "         initial_train_size = '2001-01-01 23:59:00',  # Same as len(data.loc[:end_train])\n",
    "         refit              = False\n",
    "     )\n",
    "\n",
    "results = grid_search_forecaster(\n",
    "              forecaster    = forecaster,\n",
    "              y             = data.loc[:end_val, 'y'],\n",
    "              param_grid    = param_grid,\n",
    "              lags_grid     = lags_grid,\n",
    "              cv            = cv,\n",
    "              metric        = 'mean_squared_error',\n",
    "              return_best   = True,\n",
    "              n_jobs        = 'auto',\n",
    "              verbose       = False,\n",
    "              show_progress = True\n",
    "          )\n",
    "results"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "e2b2f2e0",
   "metadata": {},
   "source": [
    "Since `return_best = True`, the forecaster object is updated with the best configuration found and trained with the whole data set. This means that the final model obtained from grid search will have the best combination of lags and hyperparameters that resulted in the highest performance metric. This final model can then be used for future predictions on new data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8a718228",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <style>\n",
       "        .container-fd347b21e15b45cbbacf5e383e4f0450 {\n",
       "            font-family: 'Arial', sans-serif;\n",
       "            font-size: 0.9em;\n",
       "            color: #333333;\n",
       "            border: 1px solid #ddd;\n",
       "            background-color: #f0f8ff;\n",
       "            padding: 5px 15px;\n",
       "            border-radius: 8px;\n",
       "            max-width: 600px;\n",
       "            #margin: auto;\n",
       "        }\n",
       "        .container-fd347b21e15b45cbbacf5e383e4f0450 h2 {\n",
       "            font-size: 1.5em;\n",
       "            color: #222222;\n",
       "            border-bottom: 2px solid #ddd;\n",
       "            padding-bottom: 5px;\n",
       "            margin-bottom: 15px;\n",
       "            margin-top: 5px;\n",
       "        }\n",
       "        .container-fd347b21e15b45cbbacf5e383e4f0450 details {\n",
       "            margin: 10px 0;\n",
       "        }\n",
       "        .container-fd347b21e15b45cbbacf5e383e4f0450 summary {\n",
       "            font-weight: bold;\n",
       "            font-size: 1.1em;\n",
       "            color: #000000;\n",
       "            cursor: pointer;\n",
       "            margin-bottom: 5px;\n",
       "            background-color: #b3dbfd;\n",
       "            padding: 5px;\n",
       "            border-radius: 5px;\n",
       "        }\n",
       "        .container-fd347b21e15b45cbbacf5e383e4f0450 summary:hover {\n",
       "            color: #000000;\n",
       "            background-color: #e0e0e0;\n",
       "        }\n",
       "        .container-fd347b21e15b45cbbacf5e383e4f0450 ul {\n",
       "            font-family: 'Courier New', monospace;\n",
       "            list-style-type: none;\n",
       "            padding-left: 20px;\n",
       "            margin: 10px 0;\n",
       "            line-height: normal;\n",
       "        }\n",
       "        .container-fd347b21e15b45cbbacf5e383e4f0450 li {\n",
       "            margin: 5px 0;\n",
       "            font-family: 'Courier New', monospace;\n",
       "        }\n",
       "        .container-fd347b21e15b45cbbacf5e383e4f0450 li strong {\n",
       "            font-weight: bold;\n",
       "            color: #444444;\n",
       "        }\n",
       "        .container-fd347b21e15b45cbbacf5e383e4f0450 li::before {\n",
       "            content: \"- \";\n",
       "            color: #666666;\n",
       "        }\n",
       "        .container-fd347b21e15b45cbbacf5e383e4f0450 a {\n",
       "            color: #001633;\n",
       "            text-decoration: none;\n",
       "        }\n",
       "        .container-fd347b21e15b45cbbacf5e383e4f0450 a:hover {\n",
       "            color: #359ccb; \n",
       "        }\n",
       "    </style>\n",
       "    \n",
       "        <div class=\"container-fd347b21e15b45cbbacf5e383e4f0450\">\n",
       "            <p style=\"font-size: 1.5em; font-weight: bold; margin-block-start: 0.83em; margin-block-end: 0.83em;\">ForecasterRecursive</p>\n",
       "            <details open>\n",
       "                <summary>General Information</summary>\n",
       "                <ul>\n",
       "                    <li><strong>Estimator:</strong> LGBMRegressor</li>\n",
       "                    <li><strong>Lags:</strong> [1 2 3]</li>\n",
       "                    <li><strong>Window features:</strong> None</li>\n",
       "                    <li><strong>Window size:</strong> 3</li>\n",
       "                    <li><strong>Series name:</strong> y</li>\n",
       "                    <li><strong>Exogenous included:</strong> False</li>\n",
       "                    <li><strong>Weight function included:</strong> False</li>\n",
       "                    <li><strong>Differentiation order:</strong> None</li>\n",
       "                    <li><strong>Creation date:</strong> 2025-11-26 15:11:12</li>\n",
       "                    <li><strong>Last fit date:</strong> 2025-11-26 15:11:15</li>\n",
       "                    <li><strong>Skforecast version:</strong> 0.19.0</li>\n",
       "                    <li><strong>Python version:</strong> 3.12.11</li>\n",
       "                    <li><strong>Forecaster id:</strong> None</li>\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Exogenous Variables</summary>\n",
       "                <ul>\n",
       "                    None\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Data Transformations</summary>\n",
       "                <ul>\n",
       "                    <li><strong>Transformer for y:</strong> None</li>\n",
       "                    <li><strong>Transformer for exog:</strong> None</li>\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Training Information</summary>\n",
       "                <ul>\n",
       "                    <li><strong>Training range:</strong> [Timestamp('1991-07-01 00:00:00'), Timestamp('2006-01-01 00:00:00')]</li>\n",
       "                    <li><strong>Training index type:</strong> DatetimeIndex</li>\n",
       "                    <li><strong>Training index frequency:</strong> <MonthBegin></li>\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Estimator Parameters</summary>\n",
       "                <ul>\n",
       "                    {'boosting_type': 'gbdt', 'class_weight': None, 'colsample_bytree': 1.0, 'importance_type': 'split', 'learning_rate': 0.1, 'max_depth': 5, 'min_child_samples': 20, 'min_child_weight': 0.001, 'min_split_gain': 0.0, 'n_estimators': 100, 'n_jobs': None, 'num_leaves': 31, 'objective': None, 'random_state': 123, 'reg_alpha': 0.0, 'reg_lambda': 0.0, 'subsample': 1.0, 'subsample_for_bin': 200000, 'subsample_freq': 0, 'verbose': -1}\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Fit Kwargs</summary>\n",
       "                <ul>\n",
       "                    {}\n",
       "                </ul>\n",
       "            </details>\n",
       "            <p>\n",
       "                <a href=\"https://skforecast.org/0.19.0/api/forecasterrecursive.html\">&#128712 <strong>API Reference</strong></a>\n",
       "                &nbsp;&nbsp;\n",
       "                <a href=\"https://skforecast.org/0.19.0/user_guides/autoregressive-forecaster.html\">&#128462 <strong>User Guide</strong></a>\n",
       "            </p>\n",
       "        </div>\n",
       "        "
      ],
      "text/plain": [
       "=================== \n",
       "ForecasterRecursive \n",
       "=================== \n",
       "Estimator: LGBMRegressor \n",
       "Lags: [1 2 3] \n",
       "Window features: None \n",
       "Window size: 3 \n",
       "Series name: y \n",
       "Exogenous included: False \n",
       "Exogenous names: None \n",
       "Transformer for y: None \n",
       "Transformer for exog: None \n",
       "Weight function included: False \n",
       "Differentiation order: None \n",
       "Training range: [Timestamp('1991-07-01 00:00:00'), Timestamp('2006-01-01 00:00:00')] \n",
       "Training index type: DatetimeIndex \n",
       "Training index frequency: <MonthBegin> \n",
       "Estimator parameters: \n",
       "    {'boosting_type': 'gbdt', 'class_weight': None, 'colsample_bytree': 1.0,\n",
       "    'importance_type': 'split', 'learning_rate': 0.1, 'max_depth': 5,\n",
       "    'min_child_samples': 20, 'min_child_weight': 0.001, 'min_split_gain': 0.0,\n",
       "    'n_estimators': 100, 'n_jobs': None, 'num_leaves': 31, 'objective': None,\n",
       "    'random_state': 123, 'reg_alpha': 0.0, 'reg_lambda': 0.0, 'subsample': 1.0,\n",
       "    'subsample_for_bin': 200000, 'subsample_freq': 0, 'verbose': -1} \n",
       "fit_kwargs: {} \n",
       "Creation date: 2025-11-26 15:11:12 \n",
       "Last fit date: 2025-11-26 15:11:15 \n",
       "Skforecast version: 0.19.0 \n",
       "Python version: 3.12.11 \n",
       "Forecaster id: None "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "forecaster"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2ab71330",
   "metadata": {},
   "source": [
    "## Random search\n",
    "\n",
    "Random search is another hyperparameter tuning strategy available in the **skforecast** library. In contrast to grid search, which tries out all possible combinations of hyperparameters and lags, randomized search samples a fixed number of values from the specified possibilities. The number of combinations that are evaluated is given by `n_iter`.\n",
    "\n",
    "It is important to note that random sampling is only applied to the model hyperparameters, but not to the lags. All lags specified by the user are evaluated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "395d1e1f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0c926dad4fab415c800c9a510e8b5d2e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "lags grid:   0%|          | 0/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1234525c2c7c4a53be876f7d5bca840a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "params grid:   0%|          | 0/5 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "`Forecaster` refitted using the best-found lags and parameters, and the whole data set: \n",
      "  Lags: [1 2 3 4 5] \n",
      "  Parameters: {'n_estimators': np.int64(96), 'max_depth': np.int64(19)}\n",
      "  Backtesting metric: 0.04313147793349785\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lags</th>\n",
       "      <th>lags_label</th>\n",
       "      <th>params</th>\n",
       "      <th>mean_squared_error</th>\n",
       "      <th>n_estimators</th>\n",
       "      <th>max_depth</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>{'n_estimators': 96, 'max_depth': 19}</td>\n",
       "      <td>0.043131</td>\n",
       "      <td>96</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>{'n_estimators': 94, 'max_depth': 28}</td>\n",
       "      <td>0.043171</td>\n",
       "      <td>94</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>{'n_estimators': 77, 'max_depth': 17}</td>\n",
       "      <td>0.043663</td>\n",
       "      <td>77</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'n_estimators': 96, 'max_depth': 19}</td>\n",
       "      <td>0.043868</td>\n",
       "      <td>96</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              lags       lags_label                                 params  \\\n",
       "0  [1, 2, 3, 4, 5]  [1, 2, 3, 4, 5]  {'n_estimators': 96, 'max_depth': 19}   \n",
       "1  [1, 2, 3, 4, 5]  [1, 2, 3, 4, 5]  {'n_estimators': 94, 'max_depth': 28}   \n",
       "2  [1, 2, 3, 4, 5]  [1, 2, 3, 4, 5]  {'n_estimators': 77, 'max_depth': 17}   \n",
       "3        [1, 2, 3]        [1, 2, 3]  {'n_estimators': 96, 'max_depth': 19}   \n",
       "\n",
       "   mean_squared_error  n_estimators  max_depth  \n",
       "0            0.043131            96         19  \n",
       "1            0.043171            94         28  \n",
       "2            0.043663            77         17  \n",
       "3            0.043868            96         19  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Random search hyperparameters and lags\n",
    "# ==============================================================================\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator = LGBMRegressor(random_state=123, verbose=-1),\n",
    "                 lags      = 10  # Placeholder, the value will be overwritten\n",
    "             )\n",
    "\n",
    "# Lags used as predictors\n",
    "lags_grid = [3, 5]\n",
    "\n",
    "# Estimator hyperparameters\n",
    "param_distributions = {\n",
    "    'n_estimators': np.arange(start=10, stop=100, step=1, dtype=int),\n",
    "    'max_depth': np.arange(start=5, stop=30, step=1, dtype=int)\n",
    "}\n",
    "\n",
    "# Folds\n",
    "cv = TimeSeriesFold(\n",
    "         steps              = 12,\n",
    "         initial_train_size = len(data.loc[:end_train]),\n",
    "         refit              = False,\n",
    "     )\n",
    "\n",
    "results = random_search_forecaster(\n",
    "              forecaster          = forecaster,\n",
    "              y                   = data.loc[:end_val, 'y'],\n",
    "              lags_grid           = lags_grid,\n",
    "              param_distributions = param_distributions,\n",
    "              cv                  = cv,\n",
    "              n_iter              = 5,\n",
    "              metric              = 'mean_squared_error',\n",
    "              return_best         = True,\n",
    "              random_state        = 123,\n",
    "              n_jobs              = 'auto',\n",
    "              verbose             = False,\n",
    "              show_progress       = True\n",
    "          )\n",
    "results.head(4)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "df764b2d",
   "metadata": {},
   "source": [
    "## Bayesian search\n",
    "\n",
    "Grid and random search can yield good results, especially when the search space is well-defined. However, these methods do not consider past results, which limits their ability to focus on the most promising regions and avoid uninformative ones.\n",
    "\n",
    "A more efficient alternative is **Bayesian optimization**, which builds a probabilistic model of the objective function, typically the validation metric (e.g. RMSE, AUC, accuracy). Based on the results observed so far, the algorithm iteratively refines the search, concentrating on regions with the highest potential. This approach reduces the number of evaluations needed by prioritizing the most relevant hyperparameter combinations. It is especially useful when the search space is large or model training is computationally expensive."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8011b4d",
   "metadata": {},
   "source": [
    "<div class=\"admonition note\" name=\"html-admonition\" style=\"background: rgba(255,145,0,.1); padding-top: 0px; padding-bottom: 6px; border-radius: 8px; border-left: 8px solid #ff9100; border-color: #ff9100; padding-left: 10px; padding-right: 10px\">\n",
    "\n",
    "<p class=\"title\">\n",
    "    <i style=\"font-size: 18px; color:#ff9100; border-color: #ff1744;\"></i>\n",
    "    <b style=\"color: #ff9100;\"> <span style=\"color: #ff9100;\">&#9888;</span> Warning</b>\n",
    "</p>\n",
    "\n",
    "The <code>lags_grid</code> argument is not required when using <code>bayesian_search_forecaster</code>. Instead, the <code>lags</code> can be included directly in the <code>search_space</code>, allowing them to be optimized jointly with the other estimator hyperparameters during the search.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "40bdbc55",
   "metadata": {},
   "source": [
    "In **skforecast**, Bayesian optimization is implemented using **Optuna** and its [`Study object`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study). The goal of the optimization is to **minimize the metric** returned by the validation strategy (either backtesting or one-step-ahead).\n",
    "\n",
    "You can customize the optimization process by passing additional arguments through the `kwargs_create_study` and `kwargs_study_optimize` parameters. These are forwarded to Optuna’s [`create_study`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.create_study.html) and [`optimize method`](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.optimize), respectively.\n",
    "\n",
    "To define the **hyperparameter search space**, the `search_space` argument must be a function that takes an Optuna [Trial object](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.trial.Trial.html#optuna.trial.Trial) and returns a dictionary of parameters to evaluate."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f7735c3",
   "metadata": {},
   "source": [
    "<div class=\"admonition note\" name=\"html-admonition\" style=\"background: rgba(255,145,0,.1); padding-top: 0px; padding-bottom: 6px; border-radius: 8px; border-left: 8px solid #ff9100; border-color: #ff9100; padding-left: 10px; padding-right: 10px\">\n",
    "\n",
    "<p class=\"title\">\n",
    "    <i style=\"font-size: 18px; color:#ff9100; border-color: #ff1744;\"></i>\n",
    "    <b style=\"color: #ff9100;\"> <span style=\"color: #ff9100;\">&#9888;</span> Warning</b>\n",
    "</p>\n",
    "\n",
    "The direction of the hyperparameter optimization can be changed by setting the <code>direction</code> argument in <code>kwargs_create_study</code> to <code>'minimize'</code> or <code>'maximize'</code>. By default, the direction is set to <code>'minimize'</code> in regression and <code>'maximize'</code> in classification tasks.\n",
    "\n",
    "```python\n",
    "kwargs_create_study = {\n",
    "    'direction': 'minimize'  # or 'maximize'\n",
    "}\n",
    "```\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "c762a37c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "91971f0feb1c47f4a312183d241bb892",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lags</th>\n",
       "      <th>params</th>\n",
       "      <th>mean_absolute_error</th>\n",
       "      <th>n_estimators</th>\n",
       "      <th>min_samples_leaf</th>\n",
       "      <th>max_features</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>{'n_estimators': 19, 'min_samples_leaf': 3, 'm...</td>\n",
       "      <td>0.126995</td>\n",
       "      <td>19</td>\n",
       "      <td>3</td>\n",
       "      <td>sqrt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'n_estimators': 15, 'min_samples_leaf': 4, 'm...</td>\n",
       "      <td>0.153278</td>\n",
       "      <td>15</td>\n",
       "      <td>4</td>\n",
       "      <td>sqrt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'n_estimators': 13, 'min_samples_leaf': 3, 'm...</td>\n",
       "      <td>0.160396</td>\n",
       "      <td>13</td>\n",
       "      <td>3</td>\n",
       "      <td>sqrt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>{'n_estimators': 14, 'min_samples_leaf': 5, 'm...</td>\n",
       "      <td>0.172366</td>\n",
       "      <td>14</td>\n",
       "      <td>5</td>\n",
       "      <td>log2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              lags                                             params  \\\n",
       "0  [1, 2, 3, 4, 5]  {'n_estimators': 19, 'min_samples_leaf': 3, 'm...   \n",
       "1        [1, 2, 3]  {'n_estimators': 15, 'min_samples_leaf': 4, 'm...   \n",
       "2        [1, 2, 3]  {'n_estimators': 13, 'min_samples_leaf': 3, 'm...   \n",
       "3  [1, 2, 3, 4, 5]  {'n_estimators': 14, 'min_samples_leaf': 5, 'm...   \n",
       "\n",
       "   mean_absolute_error  n_estimators  min_samples_leaf max_features  \n",
       "0             0.126995            19                 3         sqrt  \n",
       "1             0.153278            15                 4         sqrt  \n",
       "2             0.160396            13                 3         sqrt  \n",
       "3             0.172366            14                 5         log2  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Bayesian search hyperparameters and lags with Optuna\n",
    "# ==============================================================================\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator = LGBMRegressor(random_state=123, verbose=-1),\n",
    "                 lags      = 10  # Placeholder, the value will be overwritten\n",
    "             )\n",
    "\n",
    "\n",
    "# Search space\n",
    "def search_space(trial):\n",
    "    search_space  = {\n",
    "        'lags'            : trial.suggest_categorical('lags', [3, 5]),\n",
    "        'n_estimators'    : trial.suggest_int('n_estimators', 10, 20),\n",
    "        'min_samples_leaf': trial.suggest_int('min_samples_leaf', 1, 10),\n",
    "        'max_features'    : trial.suggest_categorical('max_features', ['log2', 'sqrt'])\n",
    "    }\n",
    "    \n",
    "    return search_space\n",
    "\n",
    "\n",
    "# Folds\n",
    "cv = TimeSeriesFold(\n",
    "         steps              = 12,\n",
    "         initial_train_size = len(data.loc[:end_train]),  # Can use a date: '2001-01-01 23:59:00'\n",
    "         refit              = False,\n",
    "     )\n",
    "\n",
    "results, best_trial = bayesian_search_forecaster(\n",
    "                          forecaster            = forecaster,\n",
    "                          y                     = data.loc[:end_val, 'y'],\n",
    "                          search_space          = search_space,\n",
    "                          cv                    = cv,\n",
    "                          metric                = 'mean_absolute_error',\n",
    "                          n_trials              = 10,\n",
    "                          random_state          = 123,\n",
    "                          return_best           = False,\n",
    "                          n_jobs                = 'auto',\n",
    "                          verbose               = False,\n",
    "                          show_progress         = True,\n",
    "                          kwargs_create_study   = {},\n",
    "                          kwargs_study_optimize = {}\n",
    "                      )\n",
    "results.head(4)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c0a970a3",
   "metadata": {},
   "source": [
    "The `best_trial` return contains the details of the trial that achieved the best result during optimization. For more information, refer to the [Study class](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "400b4ab4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "FrozenTrial(number=7, state=1, values=[0.1269945910624239], datetime_start=datetime.datetime(2025, 11, 26, 15, 11, 16, 22237), datetime_complete=datetime.datetime(2025, 11, 26, 15, 11, 16, 64236), params={'lags': 5, 'n_estimators': 19, 'min_samples_leaf': 3, 'max_features': 'sqrt'}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'lags': CategoricalDistribution(choices=(3, 5)), 'n_estimators': IntDistribution(high=20, log=False, low=10, step=1), 'min_samples_leaf': IntDistribution(high=10, log=False, low=1, step=1), 'max_features': CategoricalDistribution(choices=('log2', 'sqrt'))}, trial_id=7, value=None)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Optuna best trial in the study\n",
    "# ==============================================================================\n",
    "best_trial"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad0ee978",
   "metadata": {},
   "source": [
    "## One-step-ahead validation\n",
    "\n",
    "**One-Step-Ahead** evaluates model performance using only one-step-ahead forecasts ($t+1$). This method is faster, as it requires fewer iterations, but it only tests the model's performance in the immediate next time step. Use the <code>[OneStepAheadFold](../api/model_selection.html#skforecast.model_selection._split.OneStepAheadFold)</code> class for the one-step-ahead strategy."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2237c2bf",
   "metadata": {},
   "source": [
    "<div class=\"admonition note\" name=\"html-admonition\" style=\"background: rgba(0,191,191,.1); padding-top: 0px; padding-bottom: 6px; border-radius: 8px; border-left: 8px solid #00bfa5; border-color: #00bfa5; padding-left: 10px; padding-right: 10px;\">\n",
    "\n",
    "<p class=\"title\">\n",
    "    <i style=\"font-size: 18px; color:#00bfa5;\"></i>\n",
    "    <b style=\"color: #00bfa5;\">&#128161 Tip</b>\n",
    "</p>\n",
    "\n",
    "For a more detailed comparison of the results (**execution time** and **metric**) obtained with each strategy, visit <a href=\"../faq/parameters-search-backtesting-vs-one-step-ahead.html\">Hyperparameters and lags search: backtesting vs one-step-ahead</a>.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ac3cb60b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bcbb95ce80e746e781faa57a5fd6d18b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lags</th>\n",
       "      <th>params</th>\n",
       "      <th>mean_absolute_error</th>\n",
       "      <th>n_estimators</th>\n",
       "      <th>min_samples_leaf</th>\n",
       "      <th>max_features</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>{'n_estimators': 20, 'min_samples_leaf': 6, 'm...</td>\n",
       "      <td>0.180137</td>\n",
       "      <td>20</td>\n",
       "      <td>6</td>\n",
       "      <td>log2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>{'n_estimators': 14, 'min_samples_leaf': 5, 'm...</td>\n",
       "      <td>0.180815</td>\n",
       "      <td>14</td>\n",
       "      <td>5</td>\n",
       "      <td>log2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>{'n_estimators': 16, 'min_samples_leaf': 9, 'm...</td>\n",
       "      <td>0.187584</td>\n",
       "      <td>16</td>\n",
       "      <td>9</td>\n",
       "      <td>log2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'n_estimators': 14, 'min_samples_leaf': 7, 'm...</td>\n",
       "      <td>0.188359</td>\n",
       "      <td>14</td>\n",
       "      <td>7</td>\n",
       "      <td>log2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              lags                                             params  \\\n",
       "0  [1, 2, 3, 4, 5]  {'n_estimators': 20, 'min_samples_leaf': 6, 'm...   \n",
       "1  [1, 2, 3, 4, 5]  {'n_estimators': 14, 'min_samples_leaf': 5, 'm...   \n",
       "2  [1, 2, 3, 4, 5]  {'n_estimators': 16, 'min_samples_leaf': 9, 'm...   \n",
       "3        [1, 2, 3]  {'n_estimators': 14, 'min_samples_leaf': 7, 'm...   \n",
       "\n",
       "   mean_absolute_error  n_estimators  min_samples_leaf max_features  \n",
       "0             0.180137            20                 6         log2  \n",
       "1             0.180815            14                 5         log2  \n",
       "2             0.187584            16                 9         log2  \n",
       "3             0.188359            14                 7         log2  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Bayesian search with OneStepAheadFold\n",
    "# ==============================================================================\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator = LGBMRegressor(random_state=123, verbose=-1),\n",
    "                 lags      = 10  # Placeholder, the value will be overwritten\n",
    "             )\n",
    "\n",
    "\n",
    "# Search space\n",
    "def search_space(trial):\n",
    "    search_space  = {\n",
    "        'lags'            : trial.suggest_categorical('lags', [3, 5]),\n",
    "        'n_estimators'    : trial.suggest_int('n_estimators', 10, 20),\n",
    "        'min_samples_leaf': trial.suggest_int('min_samples_leaf', 1, 10),\n",
    "        'max_features'    : trial.suggest_categorical('max_features', ['log2', 'sqrt'])\n",
    "    }\n",
    "    \n",
    "    return search_space\n",
    "\n",
    "\n",
    "# Folds\n",
    "cv = OneStepAheadFold(\n",
    "    initial_train_size = len(data.loc[:end_train])  # Can use a date: '2001-01-01 23:59:00'\n",
    ")\n",
    "\n",
    "results, best_trial = bayesian_search_forecaster(\n",
    "                          forecaster            = forecaster,\n",
    "                          y                     = data.loc[:end_val, 'y'],\n",
    "                          search_space          = search_space,\n",
    "                          cv                    = cv,\n",
    "                          metric                = 'mean_absolute_error',\n",
    "                          n_trials              = 10,\n",
    "                          random_state          = 123,\n",
    "                          return_best           = False,\n",
    "                          n_jobs                = 'auto',\n",
    "                          verbose               = False,\n",
    "                          show_progress         = True,\n",
    "                          kwargs_create_study   = {},\n",
    "                          kwargs_study_optimize = {}\n",
    "                      )\n",
    "results.head(4)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "8279bda4",
   "metadata": {},
   "source": [
    "## Hyperparameter tuning with custom metric\n",
    "\n",
    "In addition to standard metrics such as `mean_squared_error` or `mean_absolute_error`, users can define **custom metric functions**, provided they accept the arguments `y_true` (true values), `y_pred` (predicted values) and optionally `y_train` (train values), and return a numeric value (`float` or `int`).\n",
    "\n",
    "This flexibility allows evaluating model performance under specific conditions, for example, focusing only on certain months, days, non-holiday periods, or the last step of the forecast horizon.\n",
    "\n",
    "To illustrate this, consider a scenario where a 12-month forecast is generated, but only the last three months of each year are relevant for evaluation. This can be handled by defining a custom metric function that filters the desired months before computing the error, and then passing that function to the backtesting or hyperparameter tuning process.\n",
    "\n",
    "The example below shows how to optimize model parameters using a custom metric focused on the last three months of each forecasted year.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdf534ac",
   "metadata": {},
   "source": [
    "<div class=\"admonition note\" name=\"html-admonition\" style=\"background: rgba(0,191,191,.1); padding-top: 0px; padding-bottom: 6px; border-radius: 8px; border-left: 8px solid #00bfa5; border-color: #00bfa5; padding-left: 10px; padding-right: 10px;\">\n",
    "\n",
    "<p class=\"title\">\n",
    "    <i style=\"font-size: 18px; color:#00bfa5;\"></i>\n",
    "    <b style=\"color: #00bfa5;\">&#128161 Tip</b>\n",
    "</p>\n",
    "\n",
    "More information about <b>time series forecasting metrics</b> can be found in the <a href=\"../user_guides/metrics.html\">Metrics</a> guide.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "9c7b0fd2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Custom metric\n",
    "# ==============================================================================\n",
    "def custom_metric(y_true, y_pred, y_train=None):\n",
    "    \"\"\"\n",
    "    Calculate the mean squared error using only the predicted values of the last\n",
    "    3 months of the year.\n",
    "    \"\"\"\n",
    "    mask = y_true.index.month.isin([10, 11, 12])\n",
    "    metric = mean_squared_error(y_true[mask], y_pred[mask])\n",
    "    \n",
    "    return metric"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "251660d0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e29e7385dfc74ff79fa8df2cdb3f5e3b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "lags grid:   0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "00d5877a8475409b820f30f93240a068",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "params grid:   0%|          | 0/6 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "`Forecaster` refitted using the best-found lags and parameters, and the whole data set: \n",
      "  Lags: [ 1  2  3 20] \n",
      "  Parameters: {'max_depth': 5, 'n_estimators': 100}\n",
      "  Backtesting metric: 0.0681822427249296\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lags</th>\n",
       "      <th>lags_label</th>\n",
       "      <th>params</th>\n",
       "      <th>custom_metric</th>\n",
       "      <th>max_depth</th>\n",
       "      <th>n_estimators</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 100}</td>\n",
       "      <td>0.068182</td>\n",
       "      <td>5</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 100}</td>\n",
       "      <td>0.068182</td>\n",
       "      <td>10</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 100}</td>\n",
       "      <td>0.068182</td>\n",
       "      <td>15</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 100}</td>\n",
       "      <td>0.070472</td>\n",
       "      <td>5</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            lags     lags_label                                  params  \\\n",
       "0  [1, 2, 3, 20]  [1, 2, 3, 20]   {'max_depth': 5, 'n_estimators': 100}   \n",
       "1  [1, 2, 3, 20]  [1, 2, 3, 20]  {'max_depth': 10, 'n_estimators': 100}   \n",
       "2  [1, 2, 3, 20]  [1, 2, 3, 20]  {'max_depth': 15, 'n_estimators': 100}   \n",
       "3      [1, 2, 3]      [1, 2, 3]   {'max_depth': 5, 'n_estimators': 100}   \n",
       "\n",
       "   custom_metric  max_depth  n_estimators  \n",
       "0       0.068182          5           100  \n",
       "1       0.068182         10           100  \n",
       "2       0.068182         15           100  \n",
       "3       0.070472          5           100  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Grid search hyperparameter and lags with custom metric\n",
    "# ==============================================================================\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator = LGBMRegressor(random_state=123, verbose=-1),\n",
    "                 lags      = 10  # Placeholder, the value will be overwritten\n",
    "             )\n",
    "\n",
    "# Lags used as predictors\n",
    "lags_grid = [3, 10, [1, 2, 3, 20]]\n",
    "\n",
    "# Estimator hyperparameters\n",
    "param_grid = {\n",
    "    'n_estimators': [50, 100],\n",
    "    'max_depth': [5, 10, 15]\n",
    "}\n",
    "\n",
    "# Folds\n",
    "cv = TimeSeriesFold(\n",
    "         steps              = 12,\n",
    "         initial_train_size = len(data.loc[:end_train]),\n",
    "         refit              = False,\n",
    "     )\n",
    "\n",
    "results = grid_search_forecaster(\n",
    "              forecaster    = forecaster,\n",
    "              y             = data.loc[:end_val, 'y'],\n",
    "              cv            = cv,\n",
    "              param_grid    = param_grid,\n",
    "              lags_grid     = lags_grid,\n",
    "              metric        = custom_metric,\n",
    "              return_best   = True,\n",
    "              n_jobs        = 'auto',\n",
    "              verbose       = False,\n",
    "              show_progress = True\n",
    "          )\n",
    "\n",
    "results.head(4)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3f185ba5",
   "metadata": {},
   "source": [
    "## Compare multiple metrics\n",
    "\n",
    "The functions `grid_search_forecaster`, `random_search_forecaster`, and `bayesian_search_forecaster` support the evaluation of **multiple metrics** for each forecaster configuration by passing a `list` of metric functions. This list can include both built-in metrics (e.g. `mean_squared_error`, `mean_absolute_error`) and custom-defined ones.\n",
    "\n",
    "When multiple metrics are provided, the **first metric in the list** is used to select the best model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c53bd6f4",
   "metadata": {},
   "source": [
    "<div class=\"admonition note\" name=\"html-admonition\" style=\"background: rgba(0,191,191,.1); padding-top: 0px; padding-bottom: 6px; border-radius: 8px; border-left: 8px solid #00bfa5; border-color: #00bfa5; padding-left: 10px; padding-right: 10px;\">\n",
    "\n",
    "<p class=\"title\">\n",
    "    <i style=\"font-size: 18px; color:#00bfa5;\"></i>\n",
    "    <b style=\"color: #00bfa5;\">&#128161 Tip</b>\n",
    "</p>\n",
    "\n",
    "More information about <b>time series forecasting metrics</b> can be found in the <a href=\"../user_guides/metrics.html\">Metrics</a> guide.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "d58707a8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "60e898461a0f40e0b9a79cee1d1aa544",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "lags grid:   0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "26fed8f758314669b348170b15bd7f29",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "params grid:   0%|          | 0/6 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "`Forecaster` refitted using the best-found lags and parameters, and the whole data set: \n",
      "  Lags: [1 2 3] \n",
      "  Parameters: {'max_depth': 5, 'n_estimators': 100}\n",
      "  Backtesting metric: 0.18359367014650177\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lags</th>\n",
       "      <th>lags_label</th>\n",
       "      <th>params</th>\n",
       "      <th>mean_absolute_error</th>\n",
       "      <th>mean_squared_error</th>\n",
       "      <th>custom_metric</th>\n",
       "      <th>max_depth</th>\n",
       "      <th>n_estimators</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 100}</td>\n",
       "      <td>0.183594</td>\n",
       "      <td>0.043875</td>\n",
       "      <td>0.070472</td>\n",
       "      <td>5</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 100}</td>\n",
       "      <td>0.183594</td>\n",
       "      <td>0.043875</td>\n",
       "      <td>0.070472</td>\n",
       "      <td>10</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 100}</td>\n",
       "      <td>0.183594</td>\n",
       "      <td>0.043875</td>\n",
       "      <td>0.070472</td>\n",
       "      <td>15</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>[1, 2, 3, 20]</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 100}</td>\n",
       "      <td>0.184901</td>\n",
       "      <td>0.044074</td>\n",
       "      <td>0.068182</td>\n",
       "      <td>10</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            lags     lags_label                                  params  \\\n",
       "0      [1, 2, 3]      [1, 2, 3]   {'max_depth': 5, 'n_estimators': 100}   \n",
       "1      [1, 2, 3]      [1, 2, 3]  {'max_depth': 10, 'n_estimators': 100}   \n",
       "2      [1, 2, 3]      [1, 2, 3]  {'max_depth': 15, 'n_estimators': 100}   \n",
       "3  [1, 2, 3, 20]  [1, 2, 3, 20]  {'max_depth': 10, 'n_estimators': 100}   \n",
       "\n",
       "   mean_absolute_error  mean_squared_error  custom_metric  max_depth  \\\n",
       "0             0.183594            0.043875       0.070472          5   \n",
       "1             0.183594            0.043875       0.070472         10   \n",
       "2             0.183594            0.043875       0.070472         15   \n",
       "3             0.184901            0.044074       0.068182         10   \n",
       "\n",
       "   n_estimators  \n",
       "0           100  \n",
       "1           100  \n",
       "2           100  \n",
       "3           100  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Grid search hyperparameter and lags with multiple metrics\n",
    "# ==============================================================================\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator = LGBMRegressor(random_state=123, verbose=-1),\n",
    "                 lags      = 10  # Placeholder, the value will be overwritten\n",
    "             )\n",
    "\n",
    "# Lags used as predictors\n",
    "lags_grid = [3, 10, [1, 2, 3, 20]]\n",
    "\n",
    "# Estimator hyperparameters\n",
    "param_grid = {\n",
    "    'n_estimators': [50, 100],\n",
    "    'max_depth': [5, 10, 15]\n",
    "}\n",
    "\n",
    "# Folds\n",
    "cv = TimeSeriesFold(\n",
    "         steps              = 12,\n",
    "         initial_train_size = len(data.loc[:end_train]),\n",
    "         refit              = False,\n",
    "     )\n",
    "\n",
    "results = grid_search_forecaster(\n",
    "              forecaster    = forecaster,\n",
    "              y             = data.loc[:end_val, 'y'],\n",
    "              param_grid    = param_grid,\n",
    "              lags_grid     = lags_grid,\n",
    "              cv            = cv,\n",
    "              metric        = ['mean_absolute_error', mean_squared_error, custom_metric],\n",
    "              return_best   = True,\n",
    "              n_jobs        = 'auto',\n",
    "              verbose       = False,\n",
    "              show_progress = True\n",
    "          )\n",
    "\n",
    "results.head(4)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d21a6eba",
   "metadata": {},
   "source": [
    "## Compare multiple estimators\n",
    "\n",
    "The search process can be easily extended to compare several machine learning models. This can be achieved by using a simple for loop that iterates over each estimator and applying the desired function (for example, `grid_search_forecaster`). This approach allows for a more thorough exploration and can help you select the best model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "edf210dd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Grid search for estimator: RandomForestRegressor(random_state=123)\n",
      "-------------------------\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "07e7de359c974218b9664bcd409f7899",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "lags grid:   0%|          | 0/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "81153631bbc942f3812ff97cec8fe4fd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "params grid:   0%|          | 0/4 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Grid search for estimator: LGBMRegressor(random_state=123, verbose=-1)\n",
      "-------------------------\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "afff4d30acc34758b6cb4d3e8eaa2ff4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "lags grid:   0%|          | 0/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a52d691c93a84aaaac031274a3c01ca5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "params grid:   0%|          | 0/4 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Grid search for estimator: Ridge(random_state=123)\n",
      "-------------------------\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2b96ef54daa64490bd89a03bef106d40",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "lags grid:   0%|          | 0/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fddfd88e28f54bdda1b03ad943d69860",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "params grid:   0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lags</th>\n",
       "      <th>lags_label</th>\n",
       "      <th>params</th>\n",
       "      <th>mean_squared_error</th>\n",
       "      <th>max_depth</th>\n",
       "      <th>n_estimators</th>\n",
       "      <th>model</th>\n",
       "      <th>alpha</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 50}</td>\n",
       "      <td>0.050180</td>\n",
       "      <td>5.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>LGBMRegressor</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 50}</td>\n",
       "      <td>0.050180</td>\n",
       "      <td>10.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>LGBMRegressor</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 50}</td>\n",
       "      <td>0.050907</td>\n",
       "      <td>10.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>LGBMRegressor</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 50}</td>\n",
       "      <td>0.050907</td>\n",
       "      <td>5.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>LGBMRegressor</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 20}</td>\n",
       "      <td>0.056990</td>\n",
       "      <td>5.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>LGBMRegressor</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 20}</td>\n",
       "      <td>0.056990</td>\n",
       "      <td>10.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>LGBMRegressor</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 20}</td>\n",
       "      <td>0.057542</td>\n",
       "      <td>10.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>LGBMRegressor</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>[1, 2, 3, 4, 5]</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 20}</td>\n",
       "      <td>0.057542</td>\n",
       "      <td>5.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>LGBMRegressor</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'alpha': 0.01}</td>\n",
       "      <td>0.059814</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Ridge</td>\n",
       "      <td>0.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>[1, 2, 3]</td>\n",
       "      <td>{'alpha': 0.1}</td>\n",
       "      <td>0.060078</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Ridge</td>\n",
       "      <td>0.10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              lags       lags_label                                 params  \\\n",
       "1  [1, 2, 3, 4, 5]  [1, 2, 3, 4, 5]   {'max_depth': 5, 'n_estimators': 50}   \n",
       "0  [1, 2, 3, 4, 5]  [1, 2, 3, 4, 5]  {'max_depth': 10, 'n_estimators': 50}   \n",
       "3        [1, 2, 3]        [1, 2, 3]  {'max_depth': 10, 'n_estimators': 50}   \n",
       "2        [1, 2, 3]        [1, 2, 3]   {'max_depth': 5, 'n_estimators': 50}   \n",
       "5        [1, 2, 3]        [1, 2, 3]   {'max_depth': 5, 'n_estimators': 20}   \n",
       "4        [1, 2, 3]        [1, 2, 3]  {'max_depth': 10, 'n_estimators': 20}   \n",
       "7  [1, 2, 3, 4, 5]  [1, 2, 3, 4, 5]  {'max_depth': 10, 'n_estimators': 20}   \n",
       "6  [1, 2, 3, 4, 5]  [1, 2, 3, 4, 5]   {'max_depth': 5, 'n_estimators': 20}   \n",
       "0        [1, 2, 3]        [1, 2, 3]                        {'alpha': 0.01}   \n",
       "1        [1, 2, 3]        [1, 2, 3]                         {'alpha': 0.1}   \n",
       "\n",
       "   mean_squared_error  max_depth  n_estimators          model  alpha  \n",
       "1            0.050180        5.0          50.0  LGBMRegressor    NaN  \n",
       "0            0.050180       10.0          50.0  LGBMRegressor    NaN  \n",
       "3            0.050907       10.0          50.0  LGBMRegressor    NaN  \n",
       "2            0.050907        5.0          50.0  LGBMRegressor    NaN  \n",
       "5            0.056990        5.0          20.0  LGBMRegressor    NaN  \n",
       "4            0.056990       10.0          20.0  LGBMRegressor    NaN  \n",
       "7            0.057542       10.0          20.0  LGBMRegressor    NaN  \n",
       "6            0.057542        5.0          20.0  LGBMRegressor    NaN  \n",
       "0            0.059814        NaN           NaN          Ridge   0.01  \n",
       "1            0.060078        NaN           NaN          Ridge   0.10  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Models to compare\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from lightgbm import LGBMRegressor\n",
    "from sklearn.linear_model import Ridge\n",
    "\n",
    "models = [\n",
    "    RandomForestRegressor(random_state=123), \n",
    "    LGBMRegressor(random_state=123, verbose=-1),\n",
    "    Ridge(random_state=123)\n",
    "]\n",
    "\n",
    "# Hyperparameter to search for each model\n",
    "param_grids = {\n",
    "    'RandomForestRegressor': {'n_estimators': [50, 100], 'max_depth': [5, 15]},\n",
    "    'LGBMRegressor': {'n_estimators': [20, 50], 'max_depth': [5, 10]},\n",
    "    'Ridge': {'alpha': [0.01, 0.1, 1]}\n",
    "}\n",
    "\n",
    "# Lags used as predictors\n",
    "lags_grid = [3, 5]\n",
    "\n",
    "# Folds\n",
    "cv = TimeSeriesFold(\n",
    "         steps              = 3,\n",
    "         initial_train_size = len(data.loc[:end_train]),\n",
    "         refit              = False,\n",
    "     )\n",
    "\n",
    "df_results = pd.DataFrame()\n",
    "for i, model in enumerate(models):\n",
    "\n",
    "    print(f\"Grid search for estimator: {model}\")\n",
    "    print(\"-------------------------\")\n",
    "\n",
    "    forecaster = ForecasterRecursive(\n",
    "                     estimator = model,\n",
    "                     lags      = 3\n",
    "                 )\n",
    "\n",
    "    # Estimator hyperparameters\n",
    "    param_grid = param_grids[list(param_grids)[i]]\n",
    "\n",
    "    results = grid_search_forecaster(\n",
    "                  forecaster    = forecaster,\n",
    "                  y             = data.loc[:end_val, 'y'],\n",
    "                  param_grid    = param_grid,\n",
    "                  lags_grid     = lags_grid,\n",
    "                  cv            = cv,\n",
    "                  metric        = 'mean_squared_error',\n",
    "                  return_best   = False,\n",
    "                  n_jobs        = 'auto',\n",
    "                  verbose       = False,\n",
    "                  show_progress = True\n",
    "              )\n",
    "    \n",
    "    # Create a column with model name\n",
    "    results['model'] = list(param_grids)[i]\n",
    "    \n",
    "    df_results = pd.concat([df_results, results])\n",
    "\n",
    "df_results = df_results.sort_values(by='mean_squared_error')\n",
    "df_results.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f77c791",
   "metadata": {},
   "source": [
    "## Saving results to file\n",
    "\n",
    "The results of the hyperparameter search process can be saved to a file by setting the `output_file` argument to the desired path. The results will be saved in a tab-separated values (TSV) format containing the hyperparameters, lags, and metrics of each configuration evaluated during the search. \n",
    "\n",
    "The saving process occurs after each hyperparameter evaluation, which means that if the optimization is stopped in the middle of the process, the logs of the first part of the evaluation have already been stored in the file. This can be useful for further analysis or to keep a record of the tuning process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "d7125be6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ce778f015dda40bf85718bf6a811e957",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "lags grid:   0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "64d4fad67a644caa85fdfaedbc3ed917",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "params grid:   0%|          | 0/6 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "`Forecaster` refitted using the best-found lags and parameters, and the whole data set: \n",
      "  Lags: [1 2 3] \n",
      "  Parameters: {'max_depth': 5, 'n_estimators': 100}\n",
      "  Backtesting metric: 0.04387531272712768\n"
     ]
    }
   ],
   "source": [
    "# Save results to file\n",
    "# ==============================================================================\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator = LGBMRegressor(random_state=123, verbose=-1),\n",
    "                 lags      = 10  # Placeholder, the value will be overwritten\n",
    "             )\n",
    "\n",
    "# Lags used as predictors\n",
    "lags_grid = [3, 10, [1, 2, 3, 20]]\n",
    "\n",
    "# Estimator hyperparameters\n",
    "param_grid = {\n",
    "    'n_estimators': [50, 100],\n",
    "    'max_depth': [5, 10, 15]\n",
    "}\n",
    "\n",
    "# Folds\n",
    "cv = TimeSeriesFold(\n",
    "         steps              = 12,\n",
    "         initial_train_size = len(data.loc[:end_train]),\n",
    "         refit              = False\n",
    "     )\n",
    "\n",
    "results = grid_search_forecaster(\n",
    "              forecaster    = forecaster,\n",
    "              y             = data.loc[:end_val, 'y'],\n",
    "              param_grid    = param_grid,\n",
    "              lags_grid     = lags_grid,\n",
    "              cv            = cv,\n",
    "              metric        = 'mean_squared_error',\n",
    "              return_best   = True,\n",
    "              n_jobs        = 'auto',\n",
    "              verbose       = False,\n",
    "              show_progress = True,\n",
    "              output_file   = \"results_grid_search.txt\"\n",
    "          )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "a3f34b1f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lags</th>\n",
       "      <th>lags_label</th>\n",
       "      <th>params</th>\n",
       "      <th>mean_squared_error</th>\n",
       "      <th>max_depth</th>\n",
       "      <th>n_estimators</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[1 2 3]</td>\n",
       "      <td>[1 2 3]</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 50}</td>\n",
       "      <td>0.045423</td>\n",
       "      <td>5</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[1 2 3]</td>\n",
       "      <td>[1 2 3]</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 100}</td>\n",
       "      <td>0.043875</td>\n",
       "      <td>5</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[1 2 3]</td>\n",
       "      <td>[1 2 3]</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 50}</td>\n",
       "      <td>0.045423</td>\n",
       "      <td>10</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[1 2 3]</td>\n",
       "      <td>[1 2 3]</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 100}</td>\n",
       "      <td>0.043875</td>\n",
       "      <td>10</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>[1 2 3]</td>\n",
       "      <td>[1 2 3]</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 50}</td>\n",
       "      <td>0.045423</td>\n",
       "      <td>15</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>[1 2 3]</td>\n",
       "      <td>[1 2 3]</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 100}</td>\n",
       "      <td>0.043875</td>\n",
       "      <td>15</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>[ 1  2  3  4  5  6  7  8  9 10]</td>\n",
       "      <td>[ 1  2  3  4  5  6  7  8  9 10]</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 50}</td>\n",
       "      <td>0.051399</td>\n",
       "      <td>5</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>[ 1  2  3  4  5  6  7  8  9 10]</td>\n",
       "      <td>[ 1  2  3  4  5  6  7  8  9 10]</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 100}</td>\n",
       "      <td>0.047896</td>\n",
       "      <td>5</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>[ 1  2  3  4  5  6  7  8  9 10]</td>\n",
       "      <td>[ 1  2  3  4  5  6  7  8  9 10]</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 50}</td>\n",
       "      <td>0.051399</td>\n",
       "      <td>10</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>[ 1  2  3  4  5  6  7  8  9 10]</td>\n",
       "      <td>[ 1  2  3  4  5  6  7  8  9 10]</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 100}</td>\n",
       "      <td>0.047896</td>\n",
       "      <td>10</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>[ 1  2  3  4  5  6  7  8  9 10]</td>\n",
       "      <td>[ 1  2  3  4  5  6  7  8  9 10]</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 50}</td>\n",
       "      <td>0.051399</td>\n",
       "      <td>15</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>[ 1  2  3  4  5  6  7  8  9 10]</td>\n",
       "      <td>[ 1  2  3  4  5  6  7  8  9 10]</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 100}</td>\n",
       "      <td>0.047896</td>\n",
       "      <td>15</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>[ 1  2  3 20]</td>\n",
       "      <td>[ 1  2  3 20]</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 50}</td>\n",
       "      <td>0.046221</td>\n",
       "      <td>5</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>[ 1  2  3 20]</td>\n",
       "      <td>[ 1  2  3 20]</td>\n",
       "      <td>{'max_depth': 5, 'n_estimators': 100}</td>\n",
       "      <td>0.044074</td>\n",
       "      <td>5</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>[ 1  2  3 20]</td>\n",
       "      <td>[ 1  2  3 20]</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 50}</td>\n",
       "      <td>0.046221</td>\n",
       "      <td>10</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>[ 1  2  3 20]</td>\n",
       "      <td>[ 1  2  3 20]</td>\n",
       "      <td>{'max_depth': 10, 'n_estimators': 100}</td>\n",
       "      <td>0.044074</td>\n",
       "      <td>10</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>[ 1  2  3 20]</td>\n",
       "      <td>[ 1  2  3 20]</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 50}</td>\n",
       "      <td>0.046221</td>\n",
       "      <td>15</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>[ 1  2  3 20]</td>\n",
       "      <td>[ 1  2  3 20]</td>\n",
       "      <td>{'max_depth': 15, 'n_estimators': 100}</td>\n",
       "      <td>0.044074</td>\n",
       "      <td>15</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                               lags                       lags_label  \\\n",
       "0                           [1 2 3]                          [1 2 3]   \n",
       "1                           [1 2 3]                          [1 2 3]   \n",
       "2                           [1 2 3]                          [1 2 3]   \n",
       "3                           [1 2 3]                          [1 2 3]   \n",
       "4                           [1 2 3]                          [1 2 3]   \n",
       "5                           [1 2 3]                          [1 2 3]   \n",
       "6   [ 1  2  3  4  5  6  7  8  9 10]  [ 1  2  3  4  5  6  7  8  9 10]   \n",
       "7   [ 1  2  3  4  5  6  7  8  9 10]  [ 1  2  3  4  5  6  7  8  9 10]   \n",
       "8   [ 1  2  3  4  5  6  7  8  9 10]  [ 1  2  3  4  5  6  7  8  9 10]   \n",
       "9   [ 1  2  3  4  5  6  7  8  9 10]  [ 1  2  3  4  5  6  7  8  9 10]   \n",
       "10  [ 1  2  3  4  5  6  7  8  9 10]  [ 1  2  3  4  5  6  7  8  9 10]   \n",
       "11  [ 1  2  3  4  5  6  7  8  9 10]  [ 1  2  3  4  5  6  7  8  9 10]   \n",
       "12                    [ 1  2  3 20]                    [ 1  2  3 20]   \n",
       "13                    [ 1  2  3 20]                    [ 1  2  3 20]   \n",
       "14                    [ 1  2  3 20]                    [ 1  2  3 20]   \n",
       "15                    [ 1  2  3 20]                    [ 1  2  3 20]   \n",
       "16                    [ 1  2  3 20]                    [ 1  2  3 20]   \n",
       "17                    [ 1  2  3 20]                    [ 1  2  3 20]   \n",
       "\n",
       "                                    params  mean_squared_error  max_depth  \\\n",
       "0     {'max_depth': 5, 'n_estimators': 50}            0.045423          5   \n",
       "1    {'max_depth': 5, 'n_estimators': 100}            0.043875          5   \n",
       "2    {'max_depth': 10, 'n_estimators': 50}            0.045423         10   \n",
       "3   {'max_depth': 10, 'n_estimators': 100}            0.043875         10   \n",
       "4    {'max_depth': 15, 'n_estimators': 50}            0.045423         15   \n",
       "5   {'max_depth': 15, 'n_estimators': 100}            0.043875         15   \n",
       "6     {'max_depth': 5, 'n_estimators': 50}            0.051399          5   \n",
       "7    {'max_depth': 5, 'n_estimators': 100}            0.047896          5   \n",
       "8    {'max_depth': 10, 'n_estimators': 50}            0.051399         10   \n",
       "9   {'max_depth': 10, 'n_estimators': 100}            0.047896         10   \n",
       "10   {'max_depth': 15, 'n_estimators': 50}            0.051399         15   \n",
       "11  {'max_depth': 15, 'n_estimators': 100}            0.047896         15   \n",
       "12    {'max_depth': 5, 'n_estimators': 50}            0.046221          5   \n",
       "13   {'max_depth': 5, 'n_estimators': 100}            0.044074          5   \n",
       "14   {'max_depth': 10, 'n_estimators': 50}            0.046221         10   \n",
       "15  {'max_depth': 10, 'n_estimators': 100}            0.044074         10   \n",
       "16   {'max_depth': 15, 'n_estimators': 50}            0.046221         15   \n",
       "17  {'max_depth': 15, 'n_estimators': 100}            0.044074         15   \n",
       "\n",
       "    n_estimators  \n",
       "0             50  \n",
       "1            100  \n",
       "2             50  \n",
       "3            100  \n",
       "4             50  \n",
       "5            100  \n",
       "6             50  \n",
       "7            100  \n",
       "8             50  \n",
       "9            100  \n",
       "10            50  \n",
       "11           100  \n",
       "12            50  \n",
       "13           100  \n",
       "14            50  \n",
       "15           100  \n",
       "16            50  \n",
       "17           100  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Read results file\n",
    "# ==============================================================================\n",
    "pd.read_csv(\"results_grid_search.txt\", sep=\"\\t\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "skforecast_py12",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.11"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
